Feedback device and method for providing thermal feedback using the same

ABSTRACT

A method for providing a thermal feedback, includes executing a virtual reality application providing a virtual space that includes a virtual area to which an area temperature attribute is assigned, and a virtual object to which an object temperature attributed is assigned. An area event that reflects that a player character enters the virtual area is detected. A feedback device is controlled to output thermal feedback associated to the area temperature attribute when the area event is detected, the feedback device outputting the thermal feedback using a thermoelectric element performing a thermoelectric operation. An object event reflecting the player character is influenced by the virtual object is detected. The feedback device is controlled to override the thermal feedback associated to the area temperature attribute and output thermal feedback associated to the object temperature when the object is detected while the player character is in the virtual area.

CROSS-REFERENCE TO RELATED APPLICATION

This application is continuation of U.S. patent application Ser. No.16/736,678 filed Jan. 7, 2020, which is a continuation of U.S. patentapplication Ser. No. 15/633,341 filed Jun. 26, 2017, which claims thebenefit of U.S. provisional patent application 62/415,437 filed Oct. 31,2016, which are herein incorporated by reference in their entirety

This application also claims foreign priority benefits of the filingdate of Korean Application Serial No. 10-2016-0158762, 10-2016-0158764,10-2016-0158765, 10-2016-0158767, 10-2016-0158770, 10-2016-0158774,10-2016-0158777, 10-2016-0158781, 10-2016-0158783, and 10-2016-0158785,filed on Nov. 25, 2016, and which are incorporated hereby by referencein their entirety.

TECHNICAL FIELD

The present disclosure relates generally to a feedback device and amethod for providing a thermal feedback using the feedback device. Inparticular, the present disclosure relates to a feedback device foroutputting a thermal feedback according to a reproduction of amultimedia content, a content reproduction device for reproducing themultimedia content, a system and a method for providing a thermalfeedback using the devices.

BACKGROUND

At the Consumer Electronics Show (CES) in 2016, virtual reality wasintroduced as one of the most promising future technologies. Thedevelopment of technologies such as virtual reality (VR) or augmentedreality (AR) have increased the demand for devices and methods thatenhance user experience (UX). For example, there is interest in methodsthat enhance user's immersion in the content by stimulating multiplehuman senses. VR and AR are normally confined mainly to visual andauditory senses. However, efforts are under way to include various humansenses such as olfactory and tactile sense.

Thermoelectric elements (TEs) are electrical devices that generate orabsorb heat using the Peltier effect. TEs may be used to provide athermal feedback to a user. However, the incorporation of thethermoelectric elements in VR or AR applications has been limitedbecause it is difficult to fabricate conventional thermoelectricelements using flat substrates. Thus, it is challenging to have TEs thatmake tight contact with a body part of a user.

In recent years, however, the Assignee of the present Application hassuccessfully developed flexible thermoelectric elements (FTEs) e.g., asdisclosed in Korean Application Serial No. 10-2015-0154087 filed on Nov.3, 2015. It is expected that the thermal feedback can be effectivelydelivered to users by overcoming the problems of the conventionalthermoelectric elements.

SUMMARY

The following sets forth a simplified summary of selected aspects,embodiments, and examples of the present disclosure for providing abasic understanding of the disclosure. However, the summary does notconstitute an extensive overview of all the aspects, embodiments, andexamples of the disclosure. Neither is the summary intended to identifycritical aspects or delineate the scope of the disclosure. The solepurpose of the summary is to present selected aspects, embodiments, andexamples of the disclosure in a concise form as an introduction to themore detailed description of the aspects, embodiments, and examples ofthe disclosure that follow the summary.

One aspect of the present disclosure is directed to a method forproviding a thermal feedback. The method may include executing a virtualreality application providing a virtual space, the virtual spaceincluding a virtual area to which an area temperature attribute isassigned, and a virtual object to which an object temperature attributeis assigned; detecting an area event, the area event reflecting that aplayer character enters the virtual area; controlling a feedback deviceto output a thermal feedback corresponding to the area temperatureattribute when it is determined that the area event occurs, the feedbackdevice outputting the thermal feedback using a thermoelectric elementperforming a thermoelectric operation; detecting an object event, theobject event reflecting that the player character is influenced by thevirtual object; and controlling the feedback device to override thethermal feedback corresponding to the area temperature attribute andoutput a thermal feedback corresponding to the object temperatureattribute when it is determined that the object event occurs while theplayer character is in the virtual area.

Another aspect of the present disclosure is directed to a contentreproduction device that reproduces a multimedia content. The contentreproduction device may include: a non-transitory computer readablemedium storing data; a communication module communicating with afeedback device; and a controller configured to execute instructions inthe non-transitory computer readable medium to: execute a virtualreality application providing a virtual space. The virtual space mayinclude a virtual area to which an area temperature attribute isassigned, and a virtual object to which an object temperature attributeis assigned, detect an area event, the area event reflecting that aplayer character enters the virtual area; control, via the communicationmodule, the feedback device to output a thermal feedback associated tothe area temperature attribute when it is determined that the area eventoccurs, the feedback device outputs the thermal feedback using athermoelectric element performing a thermoelectric operation, detect anobject event, the object event reflecting that the player character isinfluenced by the virtual object; and control, via the communicationmodule, the feedback device to override the thermal feedback associatedto the area temperature attribute and output a thermal feedbackassociated to the object temperature attribute when it is determinedthat the object event occurs while the player character is in thevirtual area.

Yet another aspect of the present disclosure is directed to a method forproviding a thermal feedback. The method may include executing a virtualreality application providing a virtual space. The virtual space mayinclude a global area and a local area included in the global area, theglobal area being assigned a global temperature attribute and the localarea being assigned a local temperature attribute; controlling afeedback device to output a thermal feedback corresponding to the globaltemperature attribute when it is determined that a player characterenters the global area, the feedback device outputting the thermalfeedback using a thermoelectric element performing a thermoelectricoperation; and controlling the feedback device to override the thermalfeedback corresponding to the global temperature attribute and output athermal feedback corresponding to the local temperature attribute whenit is determined that the player character enters the local area.

Another aspect of the present disclosure is directed to contentreproduction device that reproduces multimedia content. The contentreproduction device may include a non-transitory computer readablemedium storing a data; a communication module communicating with thefeedback device; a controller configured to execute instructions in thenon-transitory computer readable medium to: execute a virtual realityapplication providing a virtual space. The virtual space may include aglobal area and a local area included in the global area, the globalarea being assigned a global temperature attribute and the local areabeing assigned a local temperature attribute; control, via thecommunication module, a feedback device to output a thermal feedbackcorresponding to the global temperature attribute when it is determinedthat a player character enters the global area, the feedback deviceoutputting the thermal feedback using a thermoelectric elementperforming a thermoelectric operation, and control, via thecommunication module, the feedback device to override the thermalfeedback corresponding to the global temperature attribute and outputthe thermal feedback corresponding to the local temperature attributewhen it is determined that the player character enters the local area.

One aspect of the present disclosure is directed to a thermal feedbackproviding system and a thermal feedback providing method that provide athermal feedback by outputting thermal feedback when reproducingmultimedia content to a user.

In particular, the present disclosure aims to improve the user's contentimmersion by interlocking the audiovisual output of the multimediacontent with the thermal feedback.

One aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: reproducing a multimediacontent, wherein the multimedia content includes a video data related toa video content and a feedback data related to a thermal feedbackcorresponding to a specific scene of the video content; obtaining astart time of the thermoelectric operation, wherein the start time isset to be prior to a display time of the specific scene considering adelay duration from when the thermoelectric operation for the thermalfeedback is started to when a user senses the thermal feedback,outputting, via a display, the specific scene when the play time of themultimedia content reaches the display time; and sending, to a feedbackdevice for outputting the thermal feedback using a thermoelectricelement, a start message related to the thermal feedback when a playtime of the multimedia content reaches the start time to provide, to theuser, the thermal feedback and the specific scene together at thedisplay time.

Another aspect of the present disclosure is directed to a contentreproduction device for providing a thermal feedback, including: amemory storing a data; a communication module communicating with anexternal device; and a controller configured to: obtain, from thememory, a multimedia content and reproduce the multimedia content,wherein a multimedia content includes a video data related to a videocontent and a feedback data related to a thermal feedback correspondingto a specific scene of the video content, wherein the controller obtainsa start time of the thermoelectric operation, wherein the start time isset to be prior to a display time of the specific scene considering adelay duration from when the thermoelectric operation for the thermalfeedback is started to when a user senses the thermal feedback, outputs,via a display, the specific scene when the play time of the multimediacontent reaches the display time and sends, via the communicationmodule, a start message related to the thermal feedback to a feedbackdevice for outputting the thermal feedback using a thermoelectricelement when a play time of the multimedia content reaches the starttime to provide, to the user, the thermal feedback and the specificscene together at the display time.

Another aspect of the present disclosure is directed to a system forproviding a thermal feedback, including: a content reproduction deviceincluding: a memory storing a data, a first communication modulecommunicating with an external device, and an application controllerconfigured to obtain, from the memory, a multimedia content andreproduce the multimedia content, wherein a multimedia content includesa video data related to a video content and a feedback data related to athermal feedback corresponding to a specific scene of the video content;and a feedback device including: a second communication modulecommunicating with an external device, a thermoelectric elementperforming a thermoelectric operation for outputting the thermalfeedback, a feedback controller applying a power to the thermoelectricelement, and a contact surface which is configured to contact with abody of a user and transmits a heat generated due to the thermoelectricoperation, wherein the application controller obtains a start time ofthe thermoelectric operation, wherein the start time is set to be priorto a display time of the specific scene considering a delay durationfrom when the thermoelectric operation for the thermal feedback isstarted to when a user senses the thermal feedback, sends, via the firstcommunication module, a start message related to the thermal feedback tothe feedback device when a play time of the multimedia content reachesthe start time, and outputs, via a display, the specific scene when theplay time of the multimedia content reaches the display time, andwherein the feedback controller receives, via the second communicationmodule, the start message, applies the power to the thermoelectricelement upon the receipt of the start message to provide, to the user,the thermal feedback and the specific scene together at the displaytime.

Another aspect of the present disclosure is directed to a feedbackdevice for providing a thermal feedback related to a multimedia content,wherein the multimedia content includes a video data related to a videocontent and a feedback data related to a thermal feedback correspondingto a specific scene of the video content, the device including: athermoelectric element performing a thermoelectric operation foroutputting the thermal feedback; a feedback controller applying, to thethermoelectric element, a power for the thermoelectric operation,wherein the feedback controller applies the power to the thermoelectricelement at a start time which is set to be prior to a display time ofthe specific scene considering a delay duration from when thethermoelectric operation for the thermal feedback is started to when auser senses the thermal feedback so that the thermal feedback and thespecific scene together is provided to the user at the display time; anda contact surface being configured to contact with a body of a user,wherein a heat generated due to the thermoelectric operation istransmitted to the user through the contact surface.

Another aspect of the present disclosure is directed to a method forgenerating a multimedia content providing a thermal feedback, whereinthe thermal feedback is implemented by using a feedback device, andwherein the feedback device provides a thermal feedback due to athermoelectric operation of a thermoelectric element via a contactsurface contacting with a body of a user during a reproduction of avideo, including: obtaining a display time of a specific scene from aplay period of the video, wherein the specific scene is a scene tocorrespond to the thermal feedback; obtaining a start time of thethermoelectric operation, wherein the start time is set to be prior tothe display time considering a delay duration from when thethermoelectric operation for the thermal feedback is started to when auser senses the thermal feedback to provide, to the user, the thermalfeedback and the specific scene together at the display time; andgenerating a feedback data related to the thermal feedback, the dataincluding the start time of the thermoelectric operation.

Another aspect of the present disclosure is directed to an electronicdevice for generating a multimedia content providing a thermal feedback,wherein the thermal feedback is implemented by using a feedback device,and wherein the feedback device provides a thermal feedback due to athermoelectric operation of a thermoelectric element via a contactsurface contacting with a body of a user during a reproduction of avideo, including: a memory storing a data; and a controller configuredto: obtain a display time of a specific scene from a play period of thevideo, wherein the specific scene is a scene to correspond to thethermal feedback, obtain a start time of the thermoelectric operation,wherein the start time is set to be prior to the display timeconsidering a delay duration from when the thermoelectric operation forthe thermal feedback is started to when a user senses the thermalfeedback to provide, to the user, the thermal feedback and the specificscene together at the display time, and generate a feedback data relatedto the thermal feedback, the data including the start time of thethermoelectric operation.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to an electronic game, performed bya content reproduction device which executes the electronic game andcooperates with a feedback device outputting a thermal feedback using athermoelectric element, the method including: executing the gameincluding a player and an enemy character which attacks the player byperforming an attack action having an elemental attribute, wherein theelemental attribute includes a fire element, a cold element and anelectricity element; when the a get-hit event, reflecting that theplayer gets hit by the attack action, occurs in the game, determining atype of the thermal feedback based on the elemental attribute of theattack action, wherein the type of the thermal feedback is determined asa hot feedback when the elemental attribute of the attack action relatedto the get-hit event is the fire element, the type of the thermalfeedback is determined as a cold feedback when the elemental attributeof the attack action related to the get-hit event is the cold element,and the type of the thermal feedback is determined as a thermal grillfeedback when the elemental attribute of the attack action related tothe get-hit event is the electricity element; and controlling thefeedback device to output the thermal feedback related to the get-hitevent together with displaying a get-hit graphic so that thethermoelectric element performs a heat generating operation when thetype of the thermal feedback is the hot feedback, performs a heatabsorbing operation when the type of the thermal feedback is the coldfeedback, and performs a thermal grill operation when the type of thethermal feedback is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and a heat absorbingoperation is combined.

Another aspect of the present disclosure is directed to a contentreproduction device cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with an externaldevice; and a controller configured to: execute an electronic gameincluding a player and an enemy character which attacks the player byperforming an attack action having an elemental attribute, wherein theelemental attribute includes a fire element, a cold element and anelectricity element, when the a get-hit event, reflecting that theplayer gets hit by the attack action, occurs in the game, determines atype of the thermal feedback based on the elemental attribute of theattack action, and controls, via the communication module, the feedbackdevice to output the thermal feedback related to the get-hit eventtogether with displaying a get-hit graphic, wherein the controllerdetermines the type of the thermal feedback as a hot feedback when theelemental attribute of the attack action related to the get-hit event isthe fire element, determines the type of the thermal feedback as a coldfeedback when the elemental attribute of the attack action related tothe get-hit event is the cold element, and determines the type of thethermal feedback as a thermal grill feedback when the elementalattribute of the attack action related to the get-hit event is theelectricity element, and wherein the controller controls thethermoelectric element to perform a heat generating operation when thetype of the thermal feedback is the hot feedback, to perform a heatabsorbing operation when the type of the thermal feedback is the coldfeedback, and to perform a thermal grill operation when the type of thethermal feedback is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and a heat absorbingoperation is combined.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a feedback device, whereinthe feedback device may include a thermoelectric element which performsa thermoelectric operation including a heat generating operation, a heatabsorbing operation and a thermal grill operation in which the heatgenerating operation and the heat absorbing operation is combined, and acontact surface which is configured to contact with a body of a user,and outputs a thermal feedback by transmitting, via the contact surface,a heat generated due to the thermoelectric operation to the user, themethod including: connecting with a content reproduction deviceexecuting the game including a player and an enemy character whichattacks the player by performing an attack action having an elementalattribute, wherein the elemental attribute includes a fire element, acold element and an electricity element; when the player gets hit in thegame by the attack action of which the elemental attribute is the fireelement, outputting a hot feedback by applying a forward power to thethermoelectric element to perform the heat generating operation, whenthe player gets hit in the game by the attack action of which theelemental attribute is the cold element, outputting a cold feedback byapplying a reverse power to the thermoelectric element to perform theheat absorbing operation, and when the player gets hit in the game bythe attack action of which the elemental attribute is the electricityelement, outputting a thermal grill feedback by applying, simultaneouslyor alternatively, the forward power and the reverse power to thethermoelectric element to perform the thermal grill operation.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback corresponding to an elementalattribute in an electronic game, wherein the feedback device cooperateswith a content reproduction executing the game including a player and anenemy character which attacks the player by performing an attack actionhaving the elemental attribute, the device including: a heat outputtingmodule including a thermoelectric element which performs athermoelectric operation including a heat generating operation, a heatabsorbing operation and a thermal grill operation in which the heatgenerating operation and the heat absorbing operation is combined, apower terminal supplying a power to the thermoelectric element, and acontact surface which is configured to contact with a body of a user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated due to thethermoelectric operation to the user; and a feedback controllerconfigured to: output a hot feedback by applying a forward power to thethermoelectric element to perform the heat generating operation when theplayer gets hit in the game by the attack action of which the elementalattribute is a fire element, output a cold feedback by applying areverse power to the thermoelectric element to perform the heatabsorbing operation when the player gets hit in the game by the attackaction of which the elemental attribute is a cold element, and output athermal grill feedback by applying, simultaneously or alternatively, theforward power and the reverse power to the thermoelectric element toperform the thermal grill operation when the player gets hit in the gameby the attack action of which the elemental attribute is an electricityelement.

Another aspect of the present disclosure is directed to a system forproviding a thermal feedback, including: a content reproduction deviceexecuting an electronic game; a display displaying an image related tothe game; and a feedback device connecting with the content reproductiondevice and outputting a thermal feedback using a thermoelectric element,wherein the content reproduction device may include: a firstcommunication module communicating with the feedback device, and acontroller configured to: execute the game including a player and anenemy character which attacks the player by performing an attack actionhaving an elemental attribute, wherein the elemental attribute includesa fire element, a cold element and an electricity element, and when thea get-hit event, reflecting that the player gets hit by the attackaction, occurs in the game, display, via the display, a get-hit graphicrelated to the get-hit event, determine a type of the thermal feedbackbased on the elemental attribute of the attack action, and control, viathe first communication module, the feedback device to output thethermal feedback related to the get-hit event together with displayingthe get-hit graphic, wherein the controller determines the type of thethermal feedback as a hot feedback when the elemental attribute of theattack action related to the get-hit event is the fire element,determines the type of the thermal feedback as a cold feedback when theelemental attribute of the attack action related to the get-hit event isthe cold element, and determines the type of the thermal feedback as athermal grill feedback when the elemental attribute of the attack actionrelated to the get-hit event is the electricity element, wherein thefeedback device may include: a second communication module communicatingwith the content reproduction device, a heat outputting module includinga thermoelectric element which performs a thermoelectric operationincluding a heat generating operation, a heat absorbing operation and athermal grill operation in which the heat generating operation and theheat absorbing operation is combined, a power terminal supplying a powerto the thermoelectric element, and a contact surface which is configuredto contact with a body of a user, wherein the heat outputting moduleoutputs the thermal feedback by transmitting, via the contact surface, aheat generated due to the thermoelectric operation to the user, and afeedback controller configured to: receives, via the secondcommunication module, the type of the thermal feedback, and apply thepower to the thermoelectric element to output the thermal feedback ofthe received type, and wherein the feedback controller applies, upon thereceipt of the type indicating the hot feedback, a forward power to thethermoelectric element to perform the heat generating operation, appliesupon the receipt of the type indicating the cold feedback, a reversepower to the thermoelectric element to perform the heat absorbingoperation, and applies, upon the receipt of the type indicating thethermal grill feedback, the forward power and the reverser power,simultaneously or alternatively, to the thermoelectric element toperform the thermal grill operation.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to an electronic game, performed bya content reproduction device which executes the electronic game andcooperates with a feedback device outputting a thermal feedback using athermoelectric element, the method including: executing the gameincluding a player performing a specific action having an elementalattribute, wherein the elemental attribute includes a fire element, acold element and an electricity element; causing the player to performthe specific action according to a user command; displaying a graphicrelated to the specific action; determining a type of the thermalfeedback based on the element attribute, wherein the type is determinedas a hot feedback when the elemental attribute of the specific action isthe fire element, the type is determined as a cold feedback when theelemental attribute of the specific action is the cold element, and thetype is determined as a thermal grill feedback when the elementalattribute of the specific action is the electricity element, andcontrolling the feedback device to output the thermal feedback togetherwith displaying the graphic related to the specific action so that thethermoelectric element performs a heat generating operation when thetype is the hot feedback, performs a heat absorbing operation when thetype is the cold feedback, and performs a thermal grill operation whenthe type is the thermal grill feedback, the thermal grill operation inwhich the heat generating operation and a heat absorbing operation iscombined.

Another aspect of the present disclosure is directed to a contentreproduction device cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with an externaldevice; and a controller configured to: execute the game including aplayer performing a specific action having an elemental attribute,wherein the elemental attribute includes a fire element, a cold elementand an electricity element, cause the player to perform the specificaction according to a user command, display a graphic related to thespecific action, determine a type of the thermal feedback based on theelement attribute of the specific action, and control, via thecommunication module, the feedback device to output the thermal feedbacktogether with displaying of the graphic related to the specific action,wherein the controller determines the type as a hot feedback when theelemental attribute of the specific action is the fire element,determines the type as a cold feedback when the elemental attribute ofthe specific action is the cold element, and determines the type as athermal grill feedback when the elemental attribute of the specificaction is the electricity element, and wherein the controller controlsthe thermoelectric element to perform a heat generating operation whenthe type is the hot feedback, controls the thermoelectric element toperform a heat absorbing operation when the type is the cold feedback,and controls the thermoelectric element to perform a thermal grilloperation when the type is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and the heat absorbingoperation is combined.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a feedback device, whereinthe feedback device may include a thermoelectric element performing athermoelectric operation including at least one of a heat generatingoperation, a heat absorbing operation and a thermal grill feedback inwhich the heat generating operation and the heat absorbing operation iscombined, and a contact surface which is configured to contact with abody of a user and transmits a heat generated by the thermoelectricoperation, the method including: connecting with a content reproductiondevice executing an electronic game, wherein the game includes a playerperforming an attack action having an elemental attribute, wherein theelemental attribute includes a fire element, a cold element and anelectricity element; when the player performs the attack action of thefire element, outputting a hot feedback by applying a forward power tothe thermoelectric element to perform the heat generating operation;when the player performs the attack action of the cold element,outputting a cold feedback by applying a reverse power to thethermoelectric element to perform the heat absorbing operation; and whenthe player performs the attack action of the electricity element,outputting a thermal grill feedback by applying, simultaneously oralternatively, the forward power and the reverse power to thethermoelectric element to perform the thermal grill operation.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback correspond to an elementalattribute of an attack action, wherein the feedback device cooperateswith a content reproduction device executing an electronic game whichincludes a player performing the attack action having the elementalattribute, the elemental attribute including a fire element, a coldelement and an electricity element, the device including: an inputmodule acquiring a user input; a heat outputting module including athermoelectric element performing a thermoelectric operation including aheat generating operation, a heat absorbing operation and a thermalgrill operation in which the heat generating operation and the heatabsorbing operation is combined, a power terminal supplying a power tothe thermoelectric element, and a contact surface which is provided onone side of the thermoelectric element and is configured to contact witha body part of a user, wherein the heat outputting module outputs thethermal feedback by transmitting, via the contact surface, a heatgenerated due to the thermoelectric operation to the user; and afeedback controller configured to: output, via the heat outputtingmodule, a hot feedback by applying a forward power to the thermoelectricelement to perform the heat generating operation when the controllerreceives, via the input module, the user input instructing the attackaction having the fire element, output, via the heat outputting module,a hot feedback by applying a reverse power to the thermoelectric elementto perform the heat absorbing operation when the controller receives,via the input module, the user input instructing the attack actionhaving the cold element, wherein a current direction of the reversepower is opposite to a current direction of the forward power, andoutput, via the heat outputting module, a hot feedback by applying,simultaneously or alternatively, the forward power and reverse power tothe thermoelectric element to perform the thermal grill generatingoperation when the controller receives, via the input module, the userinput instructing the attack action having the electricity element.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device which executes the multimedia contentincluding an electronic game and a feedback application and cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, the method including: executing the multimediacontent, wherein the multimedia content includes a player and a virtualobject, implements a get-hit event in which the player gets hit by thevirtual object, and assigns, to the player, a thermal resistance relatedto the get-hit event; generating the get-hit event in the game; settingan intensity of the thermal feedback based on the get-hit event;adjusting the intensity of the thermal feedback based on the thermalresistance; and controlling the feedback device to output the thermalfeedback having the adjusted intensity.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute a multimedia content including anelectronic game and a feedback application, wherein the multimediacontent includes a player and a virtual object, implements a get-hitevent in which the player gets hit by the virtual object, and assigns,to the player, a thermal resistance related to the get-hit event, set,upon an occurrence of the get-hit event, an intensity of the thermalfeedback based on the get-hit event, adjust the intensity of the thermalfeedback based on the thermal resistance, and control, via thecommunication module, the feedback device to output the thermal feedbackhaving the adjusted intensity.

Another aspect of the present disclosure is directed to a feedbackdevice, wherein the feedback device cooperates with a contentreproduction device executing a multimedia content provided as anelectronic game or a feedback application, and wherein the multimediacontent includes a player and a virtual object, implements a get-hitevent in which the player gets hit by the virtual object, and assigns,to the player, a thermal resistance related to the get-hit event,including: a casing having a grip portion gripped by a user and formingan exterior of the feedback device; an input module receiving the userinput according to a manipulation of the user; a communication modulecommunicating with the content reproduction device; a heat outputtingmodule including a thermoelectric element performing a thermoelectricoperation, a power terminal applying a power to the thermoelectricelement, and a contact surface which is disposed on the grip portion andconfigured to contact with the user, wherein the heat outputting moduleoutputs the thermal feedback by transmitting, via the contact surface, aheat generated by the thermoelectric operation to the user; and acontroller configured to: receive, via the input module, the user input,send, via the communication module, the user input to the contentreproduction device to cause the player to act according to themanipulation of the user, receive, via the communication module, anintensity of the thermal feedback from the content reproduction device,select an operating voltage among a plurality of pre-set voltage valuesbased on the intensity of the thermal feedback, generate an operatingpower having the operating voltage, and apply the operating power to thepower terminal so that the heat outputting module outputs the thermalfeedback, and wherein the controller applies a first operating voltagewhen the get-hit event occurs, in the game, to the player who has afirst thermal resistance, and applies a second operating voltage greaterthan the first operating voltage when the get-hit event occurs, in thegame, to the player who has a second thermal resistance greater than thefirst thermal resistance.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device, wherein the content reproductiondevice executes the multimedia content including an electronic game anda feedback application and cooperates with a feedback device outputtinga thermal feedback using a thermoelectric element, including: executingthe multimedia content, wherein the multimedia content includes athermal event causing the thermal feedback, a player and an equipment towhich a thermal resistance is assigned; when the thermal event occurs,determining whether or not the player equips the equipment; when theplayer does not equip the equipment, setting an intensity of the thermalfeedback to a first intensity level; when the player equips theequipment, setting the intensity of the thermal feedback to a secondintensity level which is different from the first intensity level; andcontrolling the feedback device to output the thermal feedback accordingto the determined intensity.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device, wherein the content reproductiondevice executes the multimedia content including an electronic game anda feedback application and cooperates with a feedback device outputtinga thermal feedback using a thermoelectric element, including: executingthe multimedia content, wherein the multimedia content includes athermal event causing the thermal feedback, a player and an equipment towhich a thermal resistance is assigned; when the thermal event occurs,determining whether or not the player equips the equipment; determiningwhether or not to output the thermal feedback related to the thermalfeedback based on whether or not the player equips the equipment; andcontrolling the feedback device to output the thermal feedback only whenthe player does not equip the equipment.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content is provided as an electronic game or a feedbackapplication and includes a thermal event causing the thermal feedback, aplayer and an equipment to which a thermal resistance is assigned,determine, upon an occurrence of the thermal event, whether or not theplayer equips the equipment, set an intensity of the thermal feedback toa first intensity level when the player does not equip the equipment,set the intensity of the thermal feedback to a second intensity levelwhich is different from the first intensity level when the player equipsthe equipment, and control, via the communication module, the feedbackdevice to output the thermal feedback according to the determinedintensity.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content includes a thermal event causing the thermalfeedback, a player and an equipment to which a thermal resistance isassigned, determine, upon an occurrence of the thermal event, whether ornot the player equips the equipment, determine whether or not to outputthe thermal feedback related to the thermal feedback based on whether ornot the player equips the equipment, and control, via the communicationmodule, the feedback device to output the thermal feedback only when theplayer does not equip the equipment.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing a multimediacontent provided as an electronic game or a feedback application, andwherein the multimedia content includes a thermal event causing thethermal feedback, a player and an equipment to which a thermalresistance is assigned, including: a casing having a grip portiongripped by a user and forming an exterior of the feedback device; aninput module receiving the user input according to a manipulation of theuser; a communication module communicating with the content reproductiondevice; and a heat outputting module including a thermoelectric elementperforming a thermoelectric operation, a power terminal supplying apower to the thermoelectric element, and a contact surface which isdisposed on the grip portion and configured to contact with the user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated by thethermoelectric operation to the user; and a controller configured to:obtain, via the input module, the user input, send, via thecommunication module, the user input to the content reproduction deviceto cause the user to act according to the manipulation of the user,receive, via the communication module, a message requesting outputtingthe thermal feedback from the content reproduction device, and apply,upon the receipt of the message, the power to the power terminal so thatthe heat outputting module outputs the thermal feedback, wherein whenthe controller outputs, via the heat outputting module, the thermalfeedback upon an occurrence of the thermal event during the reproductionof the multimedia content, the controller performs a first operation inwhich whether or not to output the thermal feedback is determined basedon whether or not the player equips the equipment or a second operationin which an intensity of the thermal feedback is adjusted based onwhether or not the player equips the equipment, wherein the controllerperforms the first operation by applying an operating power to thethermoelectric element when the player does not equip the equipment andby not applying the operating power to the thermoelectric element whenthe player equips the equipment, and wherein the controller performs thesecond operation by applying a first operating power to thethermoelectric element when the player does not equip the equipment andby applying a second operating power of which a voltage magnitude issmaller than that of the first operating power.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing an electronic gameincluding a player acting according to a manipulation of a user, whereinthe player has health points and dies in the game when all of the healthpoints is exhausted; obtaining at least one of an intensity of thethermal feedback and a type of the thermal feedback according to achange of the health points during a reproduction of the game; andcontrolling the feedback device to output the thermal feedback accordingto the determined at least one of the intensity and the type, whereinthe obtaining includes at least one of: obtaining the intensity of thethermal feedback based on a change amount of the health points,obtaining the intensity of the thermal feedback based on a ratio of thechange amount to a total amount of the health points, and obtaining thetype of the thermal feedback based on whether the health points isincreased or decreased.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing an electronic gameincluding a player acting according to a manipulation of a user, whereinthe player has health points and dies in the game when all of the healthpoints is exhausted; determining at least one of whether or not tooutput the thermal feedback, an intensity of the thermal feedback and atype of the thermal feedback based on at least one of a remaining amountof the health points and a ratio of the remaining amount to a totalamount of the health point, during a reproduction of the game; andcontrolling the feedback device to output the thermal feedback accordingto the determined at least one of the whether or not to output thethermal feedback, the intensity and the type.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute an electronic game including a playeracting according to a manipulation of a user, wherein the player hashealth points and dies in the game when all of the health points isexhausted, obtain at least one of an intensity of the thermal feedbackand a type of the thermal feedback according to a change of the healthpoints during a reproduction of the game; and control, via thecommunication module, the feedback device to output the thermal feedbackaccording to the determined at least one of the intensity and the type,wherein the controller determines the intensity of the thermal feedbackbased on at least one of a change amount of the health points, anddetermines the type of the thermal feedback based on whether the healthpoints is increased or decreased.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute an electronic game including a playeracting according to a manipulation of a user, wherein the player hashealth points and dies in the game when all of the health points isexhausted, determine at least one of whether or not to output thethermal feedback, an intensity of the thermal feedback and a type of thethermal feedback based on at least one of a remaining amount of thehealth points and a ratio of the remaining amount to a total amount ofthe health point, during a reproduction of the game, and control, viathe communication module, the feedback device to output the thermalfeedback according to the determined at least one of the whether or notto output the thermal feedback, the intensity and the type.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing an electronicgame including a player acting according to a manipulation of a user,and wherein the player has health points and dies in the game when allof the health points is exhausted, including: a casing having a gripportion gripped by a user and forming an exterior of the gamingcontroller; an input module receiving the user input according to amanipulation of the user; a communication module communicating with thecontent reproduction device; a heat outputting module including athermoelectric element performing a thermoelectric operation, a powerterminal applying a power to the thermoelectric element, and a contactsurface which is disposed on the grip portion and configured to contactwith the user, wherein the heat outputting module outputs the thermalfeedback by transmitting, via the contact surface, a heat generated bythe thermoelectric operation to the user; and a controller configured toreceive, via the communication module, the user input, send, via thecommunication module, the received user input to the contentreproduction device to cause the player acts corresponding to themanipulation of the user, and apply an operating power according to achange of the health points to the power terminal so that the heatoutputting module outputs the thermal feedback corresponding to thechange of the health points, wherein the controller performs at leastone of a first operation, a second operation and a third operation,wherein the controller performs the first operation by applying a firstoperating power when a change amount of the health points or a ratio ofthe change amount to a total amount of the health points is a firstvalue and applying a second operating power of which the voltagemagnitude is greater than that of the first operating power when thechange amount or the ratio is a second value greater than the firstvalue, wherein the controller performs the second operation by applyingone of a forward power for the hot feedback and a reverser power for thecold feedback when the health points is increased and applying anotherof the forward power and the reverser power when the health points isdecreased, and wherein the controller performs the third operation byapplying a third operating power when a remaining amount of the healthpoints or a ratio of the remaining amount to the total amount is a thirdvalue and applying a fourth operating power of which the voltagemagnitude is greater than that of the third operating power when theremaining amount or the ratio of the remaining amount is a fourth valuesmaller than the third value.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing a multimedia contentprovided as an electronic game or a feedback application, wherein themultimedia content includes a virtual heat source to which a heattransferring attribute including a conduction type and a radiation typeis assigned; determining, based on the heat transferring attribute ofthe virtual heat source, a virtual heat transferring amount transferredfrom the virtual heat source to a player of the multimedia content,obtaining an intensity of the thermal feedback based on the determinedvirtual heat transferring amount; and controlling the feedback device tooutput the thermal feedback having the determined intensity. wherein thedetermining the virtual heat transferring amount includes calculatingthe virtual heat transferring amount based on a temperature value of thevirtual heat source when the heat transferring attribute of the virtualheat source is the conduction type, and calculating the virtual heattransferring amount based on the temperature value and a distancebetween the player and the virtual heat source when the heattransferring attribute of the virtual heat source is the radiation type.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute a multimedia content provided as anelectronic game or a feedback application, wherein the multimediacontent includes a virtual heat source to which a heat transferringattribute including a conduction type and a radiation type is assigned,determine, based on the heat transferring attribute of the virtual heatsource, a virtual heat transferring amount transferred from the virtualheat source to a player of the multimedia content, obtain an intensityof the thermal feedback based on the determined virtual heattransferring amount, and control, via the communication module, thefeedback device to output the thermal feedback having the determinedintensity. wherein the controller calculates the virtual heattransferring amount the virtual heat transferring amount based on atemperature value of the virtual heat source when the heat transferringattribute of the virtual heat source is the conduction type, andcalculates the virtual heat transferring amount based on the temperaturevalue and a distance between the player and the virtual heat source whenthe heat transferring attribute of the virtual heat source is theradiation type.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting the thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing an electronicgame or a feedback application which includes a virtual heat source, andoutputs the thermal feedback corresponding to a virtual heattransferring amount transferred from the virtual heat source to a playerof the game or the application, including: a heat outputting moduleincluding a thermoelectric element which performs a thermoelectricoperation including a heat generating operation, a heat absorbingoperation and a thermal grill operation in which the heat generatingoperation and the heat absorbing operation, a power terminal applying apower to the thermoelectric element, and a contact surface which isdisposed on the grip portion and configured to contact with the user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated by thethermoelectric operation to the user; and a controller configured to:control the thermoelectric element to output the thermal feedbackreflecting the virtual heat transferring amount transferred by aconduction from a first virtual heat source to the player, by applying afirst power to the power terminal when the player is contacted with thefirst virtual heat source and by stopping the application of the firstpower when the player is apart from the first virtual heat source, andcontrol the thermoelectric element to output the thermal feedbackreflecting the virtual heat transferring amount transferred by aradiation from a second virtual heat source of a different type than afirst virtual heat source, by applying a second power to the powerterminal when the player is spaced a first distance from the secondvirtual heat source and by applying a third power greater than thesecond power to the power terminal when the player is spaced a seconddistance smaller than the first distance the from the second virtualheat source.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicewhich executes a multimedia content provided as an electronic game or afeedback application and cooperates with a feedback device outputting athermal feedback using a thermoelectric element, the method including:executing the multimedia content, wherein the multimedia contentincludes a virtual object and a player, and implements an interactionbetween the player and the virtual object such as a touch or a grab, thevirtual object having a thermal attribute including a temperatureinformation and a texture information; changing an intensity of thethermal feedback according to at least one of a lasting duration of theinteraction, the temperature information and the texture information;and controlling the feedback device to output the thermal feedbackhaving the intensity.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device executes amultimedia content provided as an electronic game or a feedbackapplication, and cooperates with a feedback device outputting thethermal feedback using a thermoelectric element, including: a heatoutputting module including a thermoelectric element which performs aheat generating operation and a heat absorbing operation a powerterminal applying a power to the thermoelectric element, and a contactsurface which is disposed on the grip portion and configured to contactwith the user, wherein the heat outputting module outputs the thermalfeedback by transmitting, via the contact surface, a heat generated fromthe thermoelectric element to the user; a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content includes a virtual object and a player, andimplements an interaction between the player and the virtual object suchas a touch or a grab, the virtual object having a thermal attributeincluding a temperature information and a texture information, change anintensity of the thermal feedback according to at least one of a lastingduration of the interaction, the temperature information and the textureinformation, control the feedback device to output the thermal feedbackhaving the intensity.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing a multimediacontent provided as an electronic game or a feedback application, andwherein the multimedia content includes a virtual object and a player,and implements an interaction between the player and the virtual objectsuch as a touch or a grab, the virtual object having a thermal attributeincluding a temperature information and a texture information, thefeedback device including: a casing having a grip portion gripped by auser and forming an exterior of the feedback device; an input modulereceiving the user input according to a manipulation of the user; acommunication module communicating with the content reproduction device;a heat outputting module including a thermoelectric element performing athermoelectric operation, a power terminal applying a power to thethermoelectric element, and a contact surface which is disposed on thegrip portion and configured to contact with the user, wherein the heatoutputting module outputs the thermal feedback by transmitting, via thecontact surface, a heat generated by the thermoelectric operation to theuser; and a feedback controller configured to increase a voltagemagnitude or a current magnitude of the power applied to thethermoelectric element as a lasting duration of the interactionincreases when the interaction between the player and the virtual objectis started according to the user's operation.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing an electronic gameusing a physic engine supporting a collision processing function relatedto a collision between virtual objects, wherein the collision processingfunction includes a collision type in which a behavior of the virtualobjects is calculated considering at least a momentum of the virtualobjects and a trigger type in which one virtual object passes throughanother virtual object; determining the collision processing functionrelated to a get-hit event which occurs when a player character gets hitby the virtual object is the collision type or the trigger type;deciding an intensity of the thermal feedback based on a result of thedetermination; and controlling, based on the intensity of the thermalfeedback, an intensity of an thermoelectric operation performed by afeedback device, wherein the feedback device outputs the thermalfeedback using a thermoelectric element performing the thermoelectricoperation.

Another aspect of the present disclosure is directed to a contentreproduction device, cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with the feedbackdevice; and a controller configured to: execute an electronic game usinga physic engine supporting a collision processing function related to acollision between virtual objects, wherein the collision processingfunction includes a collision type in which a behavior of the virtualobjects is calculated considering at least a momentum of the virtualobjects and a trigger type in which one virtual object passes throughanother virtual object, determine the collision processing functionrelated to a get-hit event which occurs when a player character gets hitby the virtual object is the collision type or the trigger type, decidean intensity of the thermal feedback based on a result of thedetermination, and control, via the communication module, an intensityof an thermoelectric operation performed by the thermoelectric elementbased on the intensity of the thermal feedback.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction deviceexecuting a multimedia content and cooperating with a plurality offeedback devices outputting a thermal feedback using a thermoelectricelement, including: executing a virtual reality application providing avirtual space; obtaining a FOV (Field-Of-View) of the virtual spaceaccording to a direction of a user's sight detected from a HMD(Head-Mounted-Display); when a thermal event occurs in the virtualspace, determining at least one target device among the plurality of thefeedback devices based on an orientation of the thermal event withrespect to the FOV; and transmitting a signal instructing outputting thethermal feedback to the target feedback device.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with a plurality of feedback devicesoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback devices; a controller configured to execute a virtual realityapplication providing a virtual space, obtain a FOV (Field-Of-View) ofthe virtual space according to a direction of a user's sight detectedfrom a HMD (Head-Mounted-Display), when a thermal event occurs in thevirtual space, determine at least one target device among the pluralityof the feedback devices based on an orientation of the thermal eventwith respect to the FOV, and transmit a signal instructing outputtingthe thermal feedback to the target feedback device.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned anda virtual object to which the temperature attribute is assigned; when anarea event reflecting that a player character enters the virtual areaoccurs, control a feedback device to output the thermal feedback relatedto the area event, wherein the feedback device outputs the thermalfeedback using a thermoelectric element performing a thermoelectricoperation; detecting an occurrence of an object event reflecting that aplayer is influenced by the virtual object when the area event islasting; and when the occurrence of the object event is detected duringthe area event, controlling the feedback device to override the thermalfeedback related to the area event by the thermal feedback related tothe object event.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with at least one feedback deviceoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback device; a controller configured to: execute a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned anda virtual object to which the temperature attribute is assigned, when anarea event reflecting that a player character enters the virtual areaoccurs, control, via the communication module, the feedback device tooutput the thermal feedback related to the area event, wherein thefeedback device outputs the thermal feedback using a thermoelectricelement performing a thermoelectric operation, detect an occurrence ofan object event reflecting that a player is influenced by the virtualobject when the area event is lasting, and when the occurrence of theobject event is detected during the area event, control, via thecommunication module, the feedback device to override the thermalfeedback related to the area event by the thermal feedback related tothe object event.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned,the virtual area having a global area and a local area included in theglobal area; when a player character enters the global area, controllinga feedback device to output the thermal feedback corresponding to thetemperature attribute of the global area, wherein the feedback deviceoutputs the thermal feedback using a thermoelectric element performing athermoelectric operation; and when the player character enters the localarea, controlling the feedback device to override the thermal feedbackcorresponding to the temperature attribute of the global area with thethermal feedback corresponding to the temperature attribute of the localarea.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with at least one feedback deviceoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback device; a controller configured to: execute a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned,the virtual area having a global area and a local area included in theglobal area, when a player character enters the global area, control,via the communication module, a feedback device to output the thermalfeedback corresponding to the temperature attribute of the global area,wherein the feedback device outputs the thermal feedback using athermoelectric element performing a thermoelectric operation, and whenthe player character enters the local area, control, via thecommunication module, the feedback device to override the thermalfeedback corresponding to the temperature attribute of the global areawith the thermal feedback corresponding to the temperature attribute ofthe local area.

Another aspect of the disclosure is directed to a non-transitorycomputer-readable medium storing instructions which, when executed,cause one or more processors to perform the methods disclosed herein.The computer-readable medium may include volatile or non-volatile,magnetic, semiconductor, tape, optical, removable, non-removable, orother types of computer-readable medium or computer-readable storagedevices. For example, the computer-readable medium may be implemented asa storage unit or memory module having the computer instructions storedthereon. In some embodiments, the computer-readable medium may beimplemented as a disc or a flash drive having the computer instructionsstored thereon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a configuration of a thermal feedbackproviding system according to an embodiment of the present disclosure.

FIG. 2 is a block diagram of a configuration of a content reproductiondevice according to an embodiment of the present disclosure.

FIG. 3 is a block diagram of a configuration of an audiovisual deviceaccording to an embodiment of the present disclosure.

FIG. 4 is a block diagram of a feedback device according to anembodiment of the present disclosure.

FIG. 5 is a schematic diagram of a first implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure,

FIG. 6 is a block diagram of a first implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure

FIG. 7 is a schematic diagrams showing an exemplary form of a HMD of thefirst implementation of the thermal feedback providing system accordingto an embodiment of the present disclosure.

FIG. 8 is a schematic diagrams showing an exemplary form of a HMD of thefirst implementation of the thermal feedback providing system accordingto an embodiment of the present disclosure.

FIG. 9 is a block diagram relating to a configuration of a HMD of thefirst implementation of the thermal feedback providing system accordingto an embodiment of the present disclosure.

FIG. 10 is a schematic diagrams showing an exemplary form of an inputdevice of the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 11 is schematic diagrams showing an exemplary form of an inputdevice of the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 12 is a block diagram relating to a configuration of an inputdevice of the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure

FIG. 13 is a schematic diagram illustrating an exemplary form of inputdevice of a first implementation of a thermal feedback providing systemin accordance with an embodiment of the present disclosure.

FIG. 14 is a schematic diagram illustrating an exemplary form of inputdevice of a first implementation of a thermal feedback providing systemin accordance with an embodiment of the present disclosure.

FIG. 15 is a schematic diagram illustrating an exemplary form of inputdevice of a first implementation of a thermal feedback providing systemin accordance with an embodiment of the present disclosure.

FIG. 16 is a schematic diagram illustrating an exemplary form of inputdevice of a first implementation of a thermal feedback providing systemin accordance with an embodiment of the present disclosure.

FIG. 17 is schematic diagram illustrating an exemplary form of inputdevice of a first implementation of a thermal feedback providing systemin accordance with an embodiment of the present disclosure.

FIG. 18 is a schematic view showing an exemplary form of a wearabledevice in the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 19 is a schematic view showing an exemplary form of a wearabledevice in the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 20 is a schematic view showing an exemplary form of a wearabledevice “in the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 21 is a schematic view showing an exemplary form of a wearabledevice “in the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 22 is a schematic view showing an exemplary form of a wearabledevice “in the first implementation of the thermal feedback providingsystem according to an embodiment of the present disclosure.

FIG. 23 is a schematic diagram of the second implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 24 is a block diagram of the second implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 25 is a block diagram relating to a configuration of the smartdevice of the second implementation of the thermal feedback providingsystem according to the present application.

FIG. 26 is a schematic diagram of the third implementation of a thermalfeedback providing system according to an embodiment of the present

FIG. 27 is a schematic diagram of the third implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 28 is a schematic diagram of a fourth implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 29 is a schematic diagram of a fifth implementation of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 30 is a schematic diagram of a sixth embodiment of a thermalfeedback providing system according to an embodiment of the presentdisclosure.

FIG. 31 is a block diagram of a configuration of a heat output module1640 according to an embodiment of the present disclosure

FIG. 32 is a diagram of one embodiment of a heat output module 1640according to an embodiment of the present disclosure.

FIG. 33 is a diagram of an embodiment of a heat output module accordingto an embodiment of the present disclosure.

FIG. 34 is a diagram of an embodiment of a heat output module 1640according to an embodiment of the present disclosure.

FIG. 35 is an diagram of an embodiment of a heat output module 1640according to an embodiment of the present application.

FIG. 36 is a diagram illustrating a heat generating operation forproviding hot feedback according to an embodiment of the presentdisclosure, and

FIG. 37 is a graph relating to temperature during the hot feedback inaccordance with an embodiment of the present disclosure.

FIG. 38 is a diagram illustrating a heat absorbing operation forproviding a cold feedback according to an embodiment of the presentdisclosure,

FIG. 0.39 is a graph relating to temperature during the cold feedback inaccordance with an embodiment of the present disclosure.

FIG. 40 is a graph illustrating the degree of the intensity of thehot/cold feedback based on an adjustment of a magnitude of voltageaccording to an embodiment of the present disclosure.

FIG. 41 is a graph relating to hot/cold feedback with the sametemperature change amount according to an embodiment of the presentdisclosure.

FIG. 42 is a diagram related to a thermal grill operation according toan embodiment of the present disclosure.

FIG. 43 is a table of voltages for providing the neutral thermal grillfeedback in a voltage control manner according to an embodiment of thepresent disclosure.

FIG. 44 is a basic flowchart of a thermal feedback providing methodaccording to an embodiment of the present disclosure.

FIG. 45 is a flowchart of a first implementation of a thermal feedbackproviding method according to an embodiment of the present disclosure.

FIG. 46 is a diagram illustrating an example of thermal feedback dataused in the first implementation of the thermal feedback providingmethod according to an embodiment of the present disclosure.

FIG. 47 is a diagram of a thermal feedback output operation of a firstimplementation of a thermal feedback providing method according to anembodiment of the present disclosure.

FIG. 48 is a flowchart of the second embodiment of the thermal feedbackproviding method according to an embodiment of the present disclosure.

FIG. 49 is a flowchart of a third embodiment of a method for providing athermal feedback according to an embodiment of the present disclosure.

FIG. 50 is a diagram of a skill-thermal feedback table used in the thirdimplementation of the thermal feedback providing method according to anembodiment of the present disclosure.

FIG. 51 is a flowchart of the fourth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 52 is a diagram related to the text presentation event provided inthe fourth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure.

FIG. 53 is a flowchart of the fifth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 54 is a flowchart of the sixth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 55 is a flowchart of the seventh implementation of a method forproviding a thermal feedback according to an embodiment of the presentdisclosure.

FIG. 56 is a diagram of a game providing a timing action according tothe seventh implementation of a method for providing a thermal feedbackaccording to an embodiment of the present disclosure.

FIG. 57 is a flowchart of the eighth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 58 is a view showing a virtual space in a game in the eighthimplementation of the thermal feedback providing method according to anembodiment of the present disclosure.

FIG. 59 is a flowchart of the ninth implementation of a thermal feedbackproviding method according to an embodiment of the present disclosure.

FIG. 60 is a flowchart of the tenth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 61 is a flowchart of the eleventh implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 62 is a flow chart of the fourteenth implementation of a thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 63 is a flowchart of the fifteenth implementation of a method forproviding a thermal feedback according to an embodiment of the presentdisclosure.

FIG. 64 is a flow chart of the sixteenth implementation of a thermalfeedback providing method according to an embodiment of the presentdisclosure.

FIG. 65 is a flowchart of the seventeenth implementation of a method forproviding thermal feedback according to an embodiment of the presentdisclosure.

FIG. 66 is a view showing a thermal feedback target object according toan occurrence point of a thermal event in a first-person game accordingto the seventeenth implementation of the thermal feedback providingmethod according to an embodiment of the present disclosure.

FIG. 67 is a block diagram of an electronic device 2000 according to anembodiment of the present disclosure.

FIG. 68 is a flowchart of the first implementation of the method forgenerating multimedia content according to an embodiment of the presentdisclosure.

FIG. 69 is a flowchart of the thermal feedback providing methodaccording to an embodiment of the present disclosure.

DETAILED DESCRIPTION

One aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: reproducing a multimediacontent, wherein the multimedia content includes a video data related toa video content and a feedback data related to a thermal feedbackcorresponding to a specific scene of the video content; obtaining astart time of the thermoelectric operation, wherein the start time isset to be prior to a display time of the specific scene considering adelay duration from when the thermoelectric operation for the thermalfeedback is started to when a user senses the thermal feedback,outputting, via a display, the specific scene when the play time of themultimedia content reaches the display time; and sending, to a feedbackdevice for outputting the thermal feedback using a thermoelectricelement, a start message related to the thermal feedback when a playtime of the multimedia content reaches the start time to provide, to theuser, the thermal feedback and the specific scene together at thedisplay time.

The start time may be prior to the display time by the delay duration.

The feedback data may include an identification information indicatingthe specific scene corresponding to the thermal feedback. And theobtaining may include: obtaining the identification information from thefeedback information, obtaining the display time from the video databased on the identification information and calculating the start timebased on the display time and the delay duration.

The calculating may be performed by subtracting the delay duration fromthe display time.

The feedback data may include a output time of the thermal feedback, theoutput time being set to be same with the display time. And theobtaining may include: obtaining the output time from the feedbackinformation, and calculating the start time based on the output time andthe delay duration.

The calculating may be performed by subtracting the delay duration fromthe output time.

The thermoelectric operation may include at least one of a heatgenerating operation and a heat absorbing operation which are performedby the thermoelectric element when the power is applied thereto.

The feedback data may include a feedback type information including ahot feedback and cold feedback. And the method may further include:determining a type of the thermal feedback based on the feedback data,and obtaining the delay duration based on the type of the thermalfeedback.

The feedback data may include a feedback intensity information, and themethod may further include: determining an intensity of the thermalfeedback based on the feedback data, and obtaining the delay durationconsidering the intensity of the thermal feedback.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity, and a firstduration which is the delay duration related to the first intensity maybe smaller than a second duration which is the delay duration related tothe second intensity.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity, and a firstduration which is the delay duration related to the first intensity maybe greater than a second duration which is the delay duration related tothe second intensity.

The method may further include: obtaining, from the feedback device, adevice information identifying the feedback device; and obtaining thedelay duration based on the device information.

The method may further include receiving, from the feedback device, thedelay duration.

The video data and the feedback data may be included in a single file.

The video data and the feedback data may be included in different files.

The outputting may be performed by transmitting a video signal to anexternal device having the display.

Another aspect of the present disclosure is directed to a contentreproduction device for providing a thermal feedback, including: amemory storing a data; a communication module communicating with anexternal device; and a controller configured to: obtain, from thememory, a multimedia content and reproduce the multimedia content,wherein a multimedia content includes a video data related to a videocontent and a feedback data related to a thermal feedback correspondingto a specific scene of the video content, wherein the controller obtainsa start time of the thermoelectric operation, wherein the start time isset to be prior to a display time of the specific scene considering adelay duration from when the thermoelectric operation for the thermalfeedback is started to when a user senses the thermal feedback, outputs,via a display, the specific scene when the play time of the multimediacontent reaches the display time and sends, via the communicationmodule, a start message related to the thermal feedback to a feedbackdevice for outputting the thermal feedback using a thermoelectricelement when a play time of the multimedia content reaches the starttime to provide, to the user, the thermal feedback and the specificscene together at the display time.

The start time may be prior to the display time by the delay duration.

The feedback data may include an identification information indicatingthe specific scene corresponding to the thermal feedback, and thecontroller may obtain the identification information from the feedbackinformation, obtain the display time from the video data based on theidentification information and calculate the start time based on thedisplay time and the delay duration.

The controller may calculate the start time by subtracting the delayduration from the display time.

The feedback data may include an output time of the thermal feedback,the output time being set to be same with the display time, and thecontroller may obtain the output time from the feedback information, andcalculate the start time based on the output time and the delayduration.

The controller may calculate the start time by subtracting the delayduration from the output time.

The thermoelectric operation may include at least one of a heatgenerating operation and a heat absorbing operation which are performedby the thermoelectric element when the power is applied thereto

The feedback data may include a feedback type information including ahot feedback and cold feedback, and the controller may determine a typeof the thermal feedback based on the feedback data, and obtains thedelay duration based on the type of the thermal feedback.

The feedback data may include a feedback intensity information, and thecontroller may determine an intensity of the thermal feedback based onthe feedback data, and obtains the delay duration considering theintensity of the thermal feedback.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity, and a firstduration which is the delay duration related to the first intensity maybe smaller than a second duration which is the delay duration related tothe second intensity.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity, and a firstduration which is the delay duration related to the first intensity maybe greater than a second duration which is the delay duration related tothe second intensity.

The controller may receive, via the communication module, a deviceinformation identifying the feedback device from the feedback device,and obtain the delay duration based on the device information.

The controller may receive, via the communication module, the delayduration from the feedback device.

The memory may store a single file including the video data and thefeedback data together.

The memory may store one file including the video data and another fileincluding the feedback data.

The controller may receive, via the communication module, the multimediacontent and store the received multimedia content in the memory.

The controller may transmit, via the communication module, a videosignal to an external device having the display so that the displaydisplays the specific scene.

Another aspect of the present disclosure is directed to a system forproviding a thermal feedback, including: a content reproduction deviceincluding: a memory storing a data, a first communication modulecommunicating with an external device, and an application controllerconfigured to obtain, from the memory, a multimedia content andreproduce the multimedia content, wherein a multimedia content includesa video data related to a video content and a feedback data related to athermal feedback corresponding to a specific scene of the video content;and a feedback device including: a second communication modulecommunicating with an external device, a thermoelectric elementperforming a thermoelectric operation for outputting the thermalfeedback, a feedback controller applying a power to the thermoelectricelement, and a contact surface which is configured to contact with abody of a user and transmits a heat generated due to the thermoelectricoperation, wherein the application controller obtains a start time ofthe thermoelectric operation, wherein the start time is set to be priorto a display time of the specific scene considering a delay durationfrom when the thermoelectric operation for the thermal feedback isstarted to when a user senses the thermal feedback, sends, via the firstcommunication module, a start message related to the thermal feedback tothe feedback device when a play time of the multimedia content reachesthe start time, and outputs, via a display, the specific scene when theplay time of the multimedia content reaches the display time, andwherein the feedback controller receives, via the second communicationmodule, the start message, applies the power to the thermoelectricelement upon the receipt of the start message to provide, to the user,the thermal feedback and the specific scene together at the displaytime.

Another aspect of the present disclosure is directed to a feedbackdevice for providing a thermal feedback related to a multimedia content,wherein the multimedia content includes a video data related to a videocontent and a feedback data related to a thermal feedback correspondingto a specific scene of the video content, the device including: athermoelectric element performing a thermoelectric operation foroutputting the thermal feedback; a feedback controller applying, to thethermoelectric element, a power for the thermoelectric operation,wherein the feedback controller applies the power to the thermoelectricelement at a start time which is set to be prior to a display time ofthe specific scene considering a delay duration from when thethermoelectric operation for the thermal feedback is started to when auser senses the thermal feedback so that the thermal feedback and thespecific scene together is provided to the user at the display time; anda contact surface being configured to contact with a body of a user,wherein a heat generated due to the thermoelectric operation istransmitted to the user through the contact surface.

Another aspect of the present disclosure is directed to a method forgenerating a multimedia content providing a thermal feedback, whereinthe thermal feedback is implemented by using a feedback device, andwherein the feedback device provides a thermal feedback due to athermoelectric operation of a thermoelectric element via a contactsurface contacting with a body of a user during a reproduction of avideo, including: obtaining a display time of a specific scene from aplay period of the video, wherein the specific scene is a scene tocorrespond to the thermal feedback; obtaining a start time of thethermoelectric operation, wherein the start time is set to be prior tothe display time considering a delay duration from when thethermoelectric operation for the thermal feedback is started to when auser senses the thermal feedback to provide, to the user, the thermalfeedback and the specific scene together at the display time; andgenerating a feedback data related to the thermal feedback, the dataincluding the start time of the thermoelectric operation.

The start time may be prior to the display time by the delay duration.

In a step of the obtaining the start time, the start time may becalculated by subtracting the delay duration from the display time.

The thermoelectric operation may include at least one of a heatgenerating operation and a heat absorbing operation.

The method may further include: obtaining information on a type of thethermal feedback including a hot feedback and a cold feedback; andobtaining the delay duration considering the type of the thermalfeedback.

The method may further include: obtaining information on an intensity ofthe thermal feedback; and obtaining the delay duration considering theintensity of the thermal feedback.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity. And a firstduration which is the delay duration related to the first intensity maybe smaller than a second duration which is the delay duration related tothe second intensity.

The intensity of the thermal feedback may include a first intensity anda second intensity stronger than the first intensity. And a firstduration which is the delay duration related to the first intensity maybe greater than a second duration which is the delay duration related tothe second intensity.

The method may further include: obtaining an identification informationon the feedback device; and obtaining the delay duration considering theidentification information.

The method may further include: generating the multimedia content inform of a single file including the feedback data and a video datarelated to the video.

The method may further include: generating the multimedia content inform of multi files, a first file of the multi files including thefeedback data and a second file of the multi files including a videodata related to the video. And the first file and the second file may belinked to each other.

Another aspect of the present disclosure is directed to an electronicdevice for generating a multimedia content providing a thermal feedback,wherein the thermal feedback is implemented by using a feedback device,and wherein the feedback device provides a thermal feedback due to athermoelectric operation of a thermoelectric element via a contactsurface contacting with a body of a user during a reproduction of avideo, including: a memory storing a data; and a controller configuredto: obtain a display time of a specific scene from a play period of thevideo, wherein the specific scene is a scene to correspond to thethermal feedback, obtain a start time of the thermoelectric operation,wherein the start time is set to be prior to the display timeconsidering a delay duration from when the thermoelectric operation forthe thermal feedback is started to when a user senses the thermalfeedback to provide, to the user, the thermal feedback and the specificscene together at the display time, and generate a feedback data relatedto the thermal feedback, the data including the start time of thethermoelectric operation.

The start time may be prior to the display time by the delay duration.

The controller may calculate the start time by subtracting the delayduration from the display time.

The thermoelectric operation may include at least one of a heatgenerating operation and a heat absorbing operation.

The device may further include: an input module receiving an user input,and the controller may receive, via the input module, the user inputincluding information on a type of the thermal feedback including a hotfeedback and a cold feedback, and determine the delay durationconsidering the type of the thermal feedback.

The memory may store a matching table of the intensity and the delayduration, and the controller may obtain the delay duration correspondingto the intensity using the matching table. And the matching table mayinclude a first intensity, a second intensity stronger than the firstintensity, a first duration related to the first intensity and a secondduration related to the second intensity greater than the firstduration.

The controller may obtain an identification information on the feedbackdevice and obtain the delay duration considering the identificationinformation.

The controller may generate the multimedia content in form of a singlefile including the feedback data and a video data related to the video.

The controller may generate the multimedia content in form of multifiles, a first file of the multi files including the feedback data and asecond file of the multi files including a video data related to thevideo. And the first file and the second file may be linked to eachother

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to an electronic game, performed bya content reproduction device which executes the electronic game andcooperates with a feedback device outputting a thermal feedback using athermoelectric element, the method may include: executing the gameincluding a player and an enemy character which attacks the player byperforming an attack action having an elemental attribute, wherein theelemental attribute includes a fire element, a cold element and anelectricity element; when the a get-hit event, reflecting that theplayer gets hit by the attack action, occurs in the game, determining atype of the thermal feedback based on the elemental attribute of theattack action, wherein the type of the thermal feedback is determined asa hot feedback when the elemental attribute of the attack action relatedto the get-hit event is the fire element, the type of the thermalfeedback is determined as a cold feedback when the elemental attributeof the attack action related to the get-hit event is the cold element,and the type of the thermal feedback is determined as a thermal grillfeedback when the elemental attribute of the attack action related tothe get-hit event is the electricity element; and controlling thefeedback device to output the thermal feedback related to the get-hitevent together with displaying a get-hit graphic so that thethermoelectric element performs a heat generating operation when thetype of the thermal feedback is the hot feedback, performs a heatabsorbing operation when the type of the thermal feedback is the coldfeedback, and performs a thermal grill operation when the type of thethermal feedback is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and a heat absorbingoperation is combined.

The method further include: determining an intensity of the thermalfeedback based on at least one of an attack power of the attack action,a damage amount of the player due to the get-hit event, a ratio of thedamage amount to a total health point of the player, and a remaininghealth point of the player, and the controlling may include controllingthe feedback device to output the thermal feedback with the determinedintensity.

When the attack action is performed by using a specific skill in thegame, the elemental attribute of the attack action may be determinedbased on the elemental attribute assigned to the specific skill.

The method may further include: determining an intensity of the thermalfeedback based on at least one of a level of the specific skill, adamage amount of the specific skill, and a tier of the specific skill,wherein the tier reflects a position of the specific skill in a skilltree having the specific skill and other skills of which the elementalattribute is same with the specific skill, and the controlling mayinclude controlling the feedback device to output the thermal feedbackwith the determined intensity.

When the attack action is performed by using a melee weapon, theelemental attribute of the attack action may be determined based on theelemental attribute assigned to the melee weapon.

The method may further include: determining an intensity of the thermalfeedback based on at least one of a grade of the melee weapon and anattack power of the melee weapon, and the controlling may includecontrolling the feedback device to output the thermal feedback with thedetermined intensity.

When the attack action is performed by using a ranged weapon, theelemental attribute of the attack action may be determined based on theelemental attribute assigned to the ranged weapon or the elementalattribute assigned to a projectile of the ranged weapon.

The elemental attribute of the attack action may be determined by theelemental attribute assigned to the projectile when the projectile hasthe elemental attribute or the elemental attribute of the attack actionmay be determined by the elemental attribute assigned to the rangedweapon when the projectile does not have a the elemental attribute.

The method may further include: determining an intensity of the thermalfeedback based on at least one of a grade of the ranged weapon, anattack power of the ranged weapon, a grade of the projectile, and anattack power of the projectile, and the controlling may includecontrolling the feedback device to output the thermal feedback with thedetermined intensity.

The method may further include: causing a debuff effect related to theget-hit event to the player, and the controlling may include controllingthe feedback device to output the thermal feedback for a debuff durationof the debuff effect.

The controlling may include controlling the feedback device to decreasethe intensity of the thermal feedback as the debuff duration passes.

The electronic game may include a two dimensional type, a virtualreality type and an augmented reality type.

Another aspect of the present disclosure is directed to a contentreproduction device cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with an externaldevice; and a controller configured to: execute an electronic gameincluding a player and an enemy character which attacks the player byperforming an attack action having an elemental attribute, wherein theelemental attribute includes a fire element, a cold element and anelectricity element, when the a get-hit event, reflecting that theplayer gets hit by the attack action, occurs in the game, determines atype of the thermal feedback based on the elemental attribute of theattack action, and controls, via the communication module, the feedbackdevice to output the thermal feedback related to the get-hit eventtogether with displaying a get-hit graphic, wherein the controllerdetermines the type of the thermal feedback as a hot feedback when theelemental attribute of the attack action related to the get-hit event isthe fire element, determines the type of the thermal feedback as a coldfeedback when the elemental attribute of the attack action related tothe get-hit event is the cold element, and determines the type of thethermal feedback as a thermal grill feedback when the elementalattribute of the attack action related to the get-hit event is theelectricity element, and wherein the controller controls thethermoelectric element to perform a heat generating operation when thetype of the thermal feedback is the hot feedback, to perform a heatabsorbing operation when the type of the thermal feedback is the coldfeedback, and to perform a thermal grill operation when the type of thethermal feedback is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and a heat absorbingoperation is combined.

The controller may execute the game by loading the game from the memoryin which the game is installed.

The controller may execute the game by loading, via the communication,the game from a game server storing the game.

The device may further include: a recording medium drive reading arecording medium; and the controller may execute the game by loading,via the recording medium drive, the game from the recording mediumstoring the game.

The device may further include: a display displaying an image; and thecontroller may display, via the display, a game graphic including theget-hit graphic.

The controller may control an external display to display a game graphicincluding the get-hit graphic.

The controller may determine an intensity of the thermal feedback basedon at least one of an attack power of the attack action, a damage amountof the player due to the get-hit event, a ratio of the damage amount toa total health point of the player, and a remaining health point of theplayer, and control the feedback device to output the thermal feedbackwith the determined intensity.

When the attack action is performed by using a specific skill in thegame, the controller may determine the elemental attribute of the attackaction based on the elemental attribute assigned to the specific skill.

The controller may determine an intensity of the thermal feedback basedon at least one of a level of the specific skill, a damage amount of thespecific skill, and a tier of the specific skill, wherein the tierreflects a position of the specific skill in a skill tree having thespecific skill and other skills of which the elemental attribute is samewith the specific skill, and control the feedback device to output thethermal feedback with the determined intensity.

The controller may cause a debuff effect related to the get-hit event tothe player, and control the feedback device to output the thermalfeedback for a debuff duration of the debuff effect.

The controller may control, via the communication module, the feedbackdevice to decrease the intensity of the thermal feedback as the debuffduration passes.

The electronic game may include a two dimensional type, a virtualreality type and an augmented reality type.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a feedback device, whereinthe feedback device may include a thermoelectric element which performsa thermoelectric operation including a heat generating operation, a heatabsorbing operation and a thermal grill operation in which the heatgenerating operation and the heat absorbing operation is combined, and acontact surface which is configured to contact with a body of a user,and outputs a thermal feedback by transmitting, via the contact surface,a heat generated due to the thermoelectric operation to the user, themethod may include: connecting with a content reproduction deviceexecuting the game including a player and an enemy character whichattacks the player by performing an attack action having an elementalattribute, wherein the elemental attribute includes a fire element, acold element and an electricity element; when the player gets hit in thegame by the attack action of which the elemental attribute is the fireelement, outputting a hot feedback by applying a forward power to thethermoelectric element to perform the heat generating operation, whenthe player gets hit in the game by the attack action of which theelemental attribute is the cold element, outputting a cold feedback byapplying a reverse power to the thermoelectric element to perform theheat absorbing operation, and when the player gets hit in the game bythe attack action of which the elemental attribute is the electricityelement, outputting a thermal grill feedback by applying, simultaneouslyor alternatively, the forward power and the reverse power to thethermoelectric element to perform the thermal grill operation.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback corresponding to an elementalattribute in an electronic game, wherein the feedback device cooperateswith a content reproduction executing the game including a player and anenemy character which attacks the player by performing an attack actionhaving the elemental attribute, the device may include: a heatoutputting module including a thermoelectric element which performs athermoelectric operation including a heat generating operation, a heatabsorbing operation and a thermal grill operation in which the heatgenerating operation and the heat absorbing operation is combined, apower terminal supplying a power to the thermoelectric element, and acontact surface which is configured to contact with a body of a user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated due to thethermoelectric operation to the user; and a feedback controllerconfigured to: output a hot feedback by applying a forward power to thethermoelectric element to perform the heat generating operation when theplayer gets hit in the game by the attack action of which the elementalattribute is a fire element, output a cold feedback by applying areverse power to the thermoelectric element to perform the heatabsorbing operation when the player gets hit in the game by the attackaction of which the elemental attribute is a cold element, and output athermal grill feedback by applying, simultaneously or alternatively, theforward power and the reverse power to the thermoelectric element toperform the thermal grill operation when the player gets hit in the gameby the attack action of which the elemental attribute is an electricityelement.

The thermoelectric element may be provided as a thermoelectric couplearray having a plurality of thermoelectric couple groups which is ableto be controlled individually. And when the feedback controller mayapply the forward power to a first group including one portion of thethermoelectric groups and apply the reverse power to a second groupincluding another portion of the thermoelectric groups when the playerin the game gets hit by the attack action of which the elementalattribute is the electricity element.

Another aspect of the present disclosure is directed to a system forproviding a thermal feedback, including: a content reproduction deviceexecuting an electronic game; a display displaying an image related tothe game; and a feedback device connecting with the content reproductiondevice and outputting a thermal feedback using a thermoelectric element,wherein the content reproduction device may include: a firstcommunication module communicating with the feedback device, and acontroller configured to: execute the game including a player and anenemy character which attacks the player by performing an attack actionhaving an elemental attribute, wherein the elemental attribute includesa fire element, a cold element and an electricity element, and when thea get-hit event, reflecting that the player gets hit by the attackaction, occurs in the game, display, via the display, a get-hit graphicrelated to the get-hit event, determine a type of the thermal feedbackbased on the elemental attribute of the attack action, and control, viathe first communication module, the feedback device to output thethermal feedback related to the get-hit event together with displayingthe get-hit graphic, wherein the controller determines the type of thethermal feedback as a hot feedback when the elemental attribute of theattack action related to the get-hit event is the fire element,determines the type of the thermal feedback as a cold feedback when theelemental attribute of the attack action related to the get-hit event isthe cold element, and determines the type of the thermal feedback as athermal grill feedback when the elemental attribute of the attack actionrelated to the get-hit event is the electricity element, wherein thefeedback device may include: a second communication module communicatingwith the content reproduction device, a heat outputting module includinga thermoelectric element which performs a thermoelectric operationincluding a heat generating operation, a heat absorbing operation and athermal grill operation in which the heat generating operation and theheat absorbing operation is combined, a power terminal supplying a powerto the thermoelectric element, and a contact surface which is configuredto contact with a body of a user, wherein the heat outputting moduleoutputs the thermal feedback by transmitting, via the contact surface, aheat generated due to the thermoelectric operation to the user, and afeedback controller configured to: receives, via the secondcommunication module, the type of the thermal feedback, and apply thepower to the thermoelectric element to output the thermal feedback ofthe received type, and wherein the feedback controller applies, upon thereceipt of the type indicating the hot feedback, a forward power to thethermoelectric element to perform the heat generating operation, appliesupon the receipt of the type indicating the cold feedback, a reversepower to the thermoelectric element to perform the heat absorbingoperation, and applies, upon the receipt of the type indicating thethermal grill feedback, the forward power and the reverser power,simultaneously or alternatively, to the thermoelectric element toperform the thermal grill operation.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to an electronic game, performed bya content reproduction device which executes the electronic game andcooperates with a feedback device outputting a thermal feedback using athermoelectric element, the method may include: executing the gameincluding a player performing a specific action having an elementalattribute, wherein the elemental attribute includes a fire element, acold element and an electricity element; causing the player to performthe specific action according to a user command; displaying a graphicrelated to the specific action; determining a type of the thermalfeedback based on the element attribute, wherein the type is determinedas a hot feedback when the elemental attribute of the specific action isthe fire element, the type is determined as a cold feedback when theelemental attribute of the specific action is the cold element, and thetype is determined as a thermal grill feedback when the elementalattribute of the specific action is the electricity element, andcontrolling the feedback device to output the thermal feedback togetherwith displaying the graphic related to the specific action so that thethermoelectric element performs a heat generating operation when thetype is the hot feedback, performs a heat absorbing operation when thetype is the cold feedback, and performs a thermal grill operation whenthe type is the thermal grill feedback, the thermal grill operation inwhich the heat generating operation and a heat absorbing operation iscombined.

The specific action may include at least one of an attack actionattacking an enemy character in the game and a buffing action helping aplayer in the game.

The specific action may be an attack action attacking an enemy characterin the game. The method may further include: determining an intensity ofthe thermal feedback based on at least one of an attack power of theattack action, a skill property related to the attack action and aweapon property related to the attack action. The controlling mayinclude controlling the feedback device to output the thermal feedbackwith the determined intensity. And the skill property may include atleast one of a skill level, a skill damage and a skill tier in a skilltree including a plurality of skills of a same elemental attribute, andthe weapon property may include at least one of a weapon grade, a weaponattack power, a projectile grade and a projectile attack power.

The controlling may include controlling the feedback device to outputthe thermal feedback for a casting duration of the specific action.

The method may further include: controlling the feedback device toincrease the intensity of the thermal feedback as the casting durationpasses.

The controlling may include when a result of the specific action remainsfor a remaining duration, controlling the feedback device to maintainoutputting the thermal feedback for the remaining duration.

The method may further include: controlling the feedback device todecrease the intensity of the thermal feedback as the remaining durationpasses.

The electronic game may include a two dimensional type, a virtualreality type and an augmented reality type.

Another aspect of the present disclosure is directed to a contentreproduction device cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with an externaldevice; and a controller configured to: execute the game including aplayer performing a specific action having an elemental attribute,wherein the elemental attribute includes a fire element, a cold elementand an electricity element, cause the player to perform the specificaction according to a user command, display a graphic related to thespecific action, determine a type of the thermal feedback based on theelement attribute of the specific action, and control, via thecommunication module, the feedback device to output the thermal feedbacktogether with displaying of the graphic related to the specific action,wherein the controller determines the type as a hot feedback when theelemental attribute of the specific action is the fire element,determines the type as a cold feedback when the elemental attribute ofthe specific action is the cold element, and determines the type as athermal grill feedback when the elemental attribute of the specificaction is the electricity element, and wherein the controller controlsthe thermoelectric element to perform a heat generating operation whenthe type is the hot feedback, controls the thermoelectric element toperform a heat absorbing operation when the type is the cold feedback,and controls the thermoelectric element to perform a thermal grilloperation when the type is the thermal grill feedback, the thermal grilloperation in which the heat generating operation and the heat absorbingoperation is combined.

The specific action may include at least one of an attack actionattacking an enemy character in the game and a buffing action helping aplayer in the game.

The specific action may be an attack action attacking an enemy characterin the game. The controller may determine an intensity of the thermalfeedback based on at least one of an attack power of the attack action,a skill property related to the attack action and a weapon propertyrelated to the attack action and controls, via the communication module,the feedback device to output the thermal feedback with the determinedintensity. And the skill property may include at least one of a skilllevel, a skill damage and a skill tier in a skill tree including aplurality of skills of a same elemental attribute, and the weaponproperty may include at least one of a weapon grade, a weapon attackpower, a projectile grade and a projectile attack power.

The controller may control the feedback device to output the thermalfeedback for a casting duration of the specific action.

The controller may control the feedback device to increase the intensityof the thermal feedback as the casting duration passes.

When a result of the specific action remains for a remaining duration,the controller may control the feedback device to maintain outputtingthe thermal feedback for the remaining duration.

The controller may control the feedback device to decrease the intensityof the thermal feedback as the remaining duration passes.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a feedback device, whereinthe feedback device may include a thermoelectric element performing athermoelectric operation including at least one of a heat generatingoperation, a heat absorbing operation and a thermal grill feedback inwhich the heat generating operation and the heat absorbing operation iscombined, and a contact surface which is configured to contact with abody of a user and transmits a heat generated by the thermoelectricoperation, the method may include: connecting with a contentreproduction device executing an electronic game, wherein the gameincludes a player performing an attack action having an elementalattribute, wherein the elemental attribute includes a fire element, acold element and an electricity element; when the player performs theattack action of the fire element, outputting a hot feedback by applyinga forward power to the thermoelectric element to perform the heatgenerating operation; when the player performs the attack action of thecold element, outputting a cold feedback by applying a reverse power tothe thermoelectric element to perform the heat absorbing operation; andwhen the player performs the attack action of the electricity element,outputting a thermal grill feedback by applying, simultaneously oralternatively, the forward power and the reverse power to thethermoelectric element to perform the thermal grill operation.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback correspond to an elementalattribute of an attack action, wherein the feedback device cooperateswith a content reproduction device executing an electronic game whichincludes a player performing the attack action having the elementalattribute, the elemental attribute including a fire element, a coldelement and an electricity element, the device may include: an inputmodule acquiring a user input; a heat outputting module including athermoelectric element performing a thermoelectric operation including aheat generating operation, a heat absorbing operation and a thermalgrill operation in which the heat generating operation and the heatabsorbing operation is combined, a power terminal supplying a power tothe thermoelectric element, and a contact surface which is provided onone side of the thermoelectric element and is configured to contact witha body part of a user, wherein the heat outputting module outputs thethermal feedback by transmitting, via the contact surface, a heatgenerated due to the thermoelectric operation to the user; and afeedback controller configured to: output, via the heat outputtingmodule, a hot feedback by applying a forward power to the thermoelectricelement to perform the heat generating operation when the controllerreceives, via the input module, the user input instructing the attackaction having the fire element, output, via the heat outputting module,a hot feedback by applying a reverse power to the thermoelectric elementto perform the heat absorbing operation when the controller receives,via the input module, the user input instructing the attack actionhaving the cold element, wherein a current direction of the reversepower is opposite to a current direction of the forward power, andoutput, via the heat outputting module, a hot feedback by applying,simultaneously or alternatively, the forward power and reverse power tothe thermoelectric element to perform the thermal grill generatingoperation when the controller receives, via the input module, the userinput instructing the attack action having the electricity element.

The thermoelectric element may be provided as a thermoelectric couplearray having a plurality of thermoelectric couple groups which are ableto be controlled individually. And the feedback controller may apply,upon the receipt of the user input instructing the attack action havingthe electricity element, the forward power to a first group being oneportion of the thermoelectric couple groups and the reverse power to asecond group being another portion of the thermoelectric couple groups.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device which executes the multimedia contentincluding an electronic game and a feedback application and cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, the method may include: executing the multimediacontent, wherein the multimedia content includes a player and a virtualobject, implements a get-hit event in which the player gets hit by thevirtual object, and assigns, to the player, a thermal resistance relatedto the get-hit event; generating the get-hit event in the game; settingan intensity of the thermal feedback based on the get-hit event;adjusting the intensity of the thermal feedback based on the thermalresistance; and controlling the feedback device to output the thermalfeedback having the adjusted intensity.

The get-hit event may include at least one of a hot-hit event and acold-hit event, and the thermal feedback may include at least of a hotfeedback and a cold feedback. The method may further include:determining a type of the thermal feedback based on a type of theget-hit event. And the controlling may include controlling the feedbackdevice to output the thermal feedback of the determined type of thethermal feedback.

The thermal resistance may include at least one of a hot resistancecorresponding to the hot-hit event and a cold resistance correspondingto the cold-hit event. And the adjusting may include adjusting theintensity of the thermal feedback based on the thermal resistancecorresponding to the type of the generated get-hit event.

The method may further include: calculating the thermal resistance basedon a thermal resistance assigned to the player and a thermal resistanceassigned to an equipment equipped by the player.

The adjusting may include reducing the intensity of the thermalfeedback.

The intensity of the thermal feedback may include a plurality ofintensity levels. The setting may include obtaining, from the pluralityof the intensity levels, a first intensity level as the intensity of thethermal feedback related to the get-hit event. And the adjusting mayinclude obtaining, from the plurality of the intensity levels, a secondintensity level lower than the first intensity level.

The adjusting may include determining the second intensity based on thefirst intensity and the thermal resistance.

The obtaining the second intensity level may be performed when the firstintensity level is not a lowest intensity level among the plurality ofthe intensity levels. And the adjusting may further include maintainingthe first intensity level as the intensity of the thermal feedbackrelated to the get-hit event when the first intensity level is thelowest intensity level among the plurality of the intensity levels.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute a multimedia content including anelectronic game and a feedback application, wherein the multimediacontent includes a player and a virtual object, implements a get-hitevent in which the player gets hit by the virtual object, and assigns,to the player, a thermal resistance related to the get-hit event, set,upon an occurrence of the get-hit event, an intensity of the thermalfeedback based on the get-hit event, adjust the intensity of the thermalfeedback based on the thermal resistance, and control, via thecommunication module, the feedback device to output the thermal feedbackhaving the adjusted intensity.

The get-hit event may include at least one of a hot-hit event and acold-hit event, and the thermal feedback may include at least of a hotfeedback and a cold feedback. And the controller may determine a type ofthe thermal feedback based on a type of the get-hit event, and controlthe feedback device to output the thermal feedback of the determinedtype of the thermal feedback.

The thermal resistance may include at least one of a hot resistancecorresponding to the hot-hit event and a cold resistance correspondingto the cold-hit event. And the controller may adjust the intensity ofthe thermal feedback based on the thermal resistance corresponding tothe type of the generated get-hit event.

The controller may calculate the thermal resistance based on a thermalresistance assigned to the player and a thermal resistance assigned toan equipment equipped by the player.

The controller may adjust the intensity of the thermal feedback byreducing the intensity of the thermal feedback considering the thermalresistance.

The intensity of the thermal feedback may include a plurality ofintensity levels. The controller may set the intensity of the thermalfeedback related to the get-hit event to a first intensity among theplurality of the intensity levels, and adjust the intensity of thethermal feedback related to the get-hit event to a second intensitylevel, which is lower than the first intensity level, among theplurality of the intensity levels.

The controller may determine the second intensity based on the firstintensity and the thermal resistance.

The controller may adjust the intensity of the thermal feedback relatedto the get-hit event to the second intensity level when the firstintensity level is not a lowest intensity level among the plurality ofthe intensity levels, and maintain the first intensity level as theintensity of the thermal feedback related to the get-hit event when thefirst intensity level is the lowest intensity level among the pluralityof the intensity levels.

Another aspect of the present disclosure is directed to a feedbackdevice, wherein the feedback device cooperates with a contentreproduction device executing a multimedia content provided as anelectronic game or a feedback application, and wherein the multimediacontent includes a player and a virtual object, implements a get-hitevent in which the player gets hit by the virtual object, and assigns,to the player, a thermal resistance related to the get-hit event,including: a casing having a grip portion gripped by a user and formingan exterior of the feedback device; an input module receiving the userinput according to a manipulation of the user; a communication modulecommunicating with the content reproduction device; a heat outputtingmodule including a thermoelectric element performing a thermoelectricoperation, a power terminal applying a power to the thermoelectricelement, and a contact surface which is disposed on the grip portion andconfigured to contact with the user, wherein the heat outputting moduleoutputs the thermal feedback by transmitting, via the contact surface, aheat generated by the thermoelectric operation to the user; and acontroller configured to: receive, via the input module, the user input,send, via the communication module, the user input to the contentreproduction device to cause the player to act according to themanipulation of the user, receive, via the communication module, anintensity of the thermal feedback from the content reproduction device,select an operating voltage among a plurality of pre-set voltage valuesbased on the intensity of the thermal feedback, generate an operatingpower having the operating voltage, and apply the operating power to thepower terminal so that the heat outputting module outputs the thermalfeedback, and wherein the controller applies a first operating voltagewhen the get-hit event occurs, in the game, to the player who has afirst thermal resistance, and applies a second operating voltage greaterthan the first operating voltage when the get-hit event occurs, in thegame, to the player who has a second thermal resistance greater than thefirst thermal resistance.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device, wherein the content reproductiondevice executes the multimedia content including an electronic game anda feedback application and cooperates with a feedback device outputtinga thermal feedback using a thermoelectric element, including: executingthe multimedia content, wherein the multimedia content includes athermal event causing the thermal feedback, a player and an equipment towhich a thermal resistance is assigned; when the thermal event occurs,determining whether or not the player equips the equipment; when theplayer does not equip the equipment, setting an intensity of the thermalfeedback to a first intensity level; when the player equips theequipment, setting the intensity of the thermal feedback to a secondintensity level which is different from the first intensity level; andcontrolling the feedback device to output the thermal feedback accordingto the determined intensity.

The second intensity level may be smaller than the first intensitylevel.

The second intensity level may be calculated based on the firstintensity level and the thermal resistance of the equipment.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback related to a multimedia content, performedby a content reproduction device, wherein the content reproductiondevice executes the multimedia content including an electronic game anda feedback application and cooperates with a feedback device outputtinga thermal feedback using a thermoelectric element, including: executingthe multimedia content, wherein the multimedia content includes athermal event causing the thermal feedback, a player and an equipment towhich a thermal resistance is assigned; when the thermal event occurs,determining whether or not the player equips the equipment; determiningwhether or not to output the thermal feedback related to the thermalfeedback based on whether or not the player equips the equipment; andcontrolling the feedback device to output the thermal feedback only whenthe player does not equip the equipment.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content is provided as an electronic game or a feedbackapplication and includes a thermal event causing the thermal feedback, aplayer and an equipment to which a thermal resistance is assigned,determine, upon an occurrence of the thermal event, whether or not theplayer equips the equipment, set an intensity of the thermal feedback toa first intensity level when the player does not equip the equipment,set the intensity of the thermal feedback to a second intensity levelwhich is different from the first intensity level when the player equipsthe equipment, and control, via the communication module, the feedbackdevice to output the thermal feedback according to the determinedintensity.

The second intensity level may be smaller than the first intensitylevel.

The controller may calculate the second intensity level based on thefirst intensity level and the thermal resistance assigned to theequipment.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting a thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content includes a thermal event causing the thermalfeedback, a player and an equipment to which a thermal resistance isassigned, determine, upon an occurrence of the thermal event, whether ornot the player equips the equipment, determine whether or not to outputthe thermal feedback related to the thermal feedback based on whether ornot the player equips the equipment, and control, via the communicationmodule, the feedback device to output the thermal feedback only when theplayer does not equip the equipment.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing a multimediacontent provided as an electronic game or a feedback application, andwherein the multimedia content includes a thermal event causing thethermal feedback, a player and an equipment to which a thermalresistance is assigned, including: a casing having a grip portiongripped by a user and forming an exterior of the feedback device; aninput module receiving the user input according to a manipulation of theuser; a communication module communicating with the content reproductiondevice; and a heat outputting module including a thermoelectric elementperforming a thermoelectric operation, a power terminal supplying apower to the thermoelectric element, and a contact surface which isdisposed on the grip portion and configured to contact with the user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated by thethermoelectric operation to the user; and a controller configured to:obtain, via the input module, the user input, send, via thecommunication module, the user input to the content reproduction deviceto cause the user to act according to the manipulation of the user,receive, via the communication module, a message requesting outputtingthe thermal feedback from the content reproduction device, and apply,upon the receipt of the message, the power to the power terminal so thatthe heat outputting module outputs the thermal feedback, wherein whenthe controller outputs, via the heat outputting module, the thermalfeedback upon an occurrence of the thermal event during the reproductionof the multimedia content, the controller performs a first operation inwhich whether or not to output the thermal feedback is determined basedon whether or not the player equips the equipment or a second operationin which an intensity of the thermal feedback is adjusted based onwhether or not the player equips the equipment, wherein the controllerperforms the first operation by applying an operating power to thethermoelectric element when the player does not equip the equipment andby not applying the operating power to the thermoelectric element whenthe player equips the equipment, and wherein the controller performs thesecond operation by applying a first operating power to thethermoelectric element when the player does not equip the equipment andby applying a second operating power of which a voltage magnitude issmaller than that of the first operating power.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing an electronic gameincluding a player acting according to a manipulation of a user, whereinthe player has health points and dies in the game when all of the healthpoints is exhausted; obtaining at least one of an intensity of thethermal feedback and a type of the thermal feedback according to achange of the health points during a reproduction of the game; andcontrolling the feedback device to output the thermal feedback accordingto the determined at least one of the intensity and the type, whereinthe obtaining includes at least one of: obtaining the intensity of thethermal feedback based on a change amount of the health points,obtaining the intensity of the thermal feedback based on a ratio of thechange amount to a total amount of the health points, and obtaining thetype of the thermal feedback based on whether the health points isincreased or decreased.

In a step of the obtaining the intensity of the thermal feedback basedon the change amount of the health points, a first intensity may beobtained when the change amount is a first value and a second intensitygreater than the first intensity may be obtained when the change amountis a second value greater than the first value.

In a step of the obtaining the intensity of the thermal feedback basedon the ratio of the change amount to the total amount of the healthpoints, a first intensity may be obtained when the ratio is a firstvalue and a second intensity greater than the first intensity may beobtained when the ratio is a second value greater than the first value.

In a step of the obtaining the intensity of the thermal feedback basedon whether the health points is increased or decreased, one of a hotfeedback and a cold feedback may be obtained as the type of the thermalfeedback when the health points is increased and another of the hotfeedback and the cold feedback may be obtained as the type of thethermal feedback when the health points is decreased.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing an electronic gameincluding a player acting according to a manipulation of a user, whereinthe player has health points and dies in the game when all of the healthpoints is exhausted; determining at least one of whether or not tooutput the thermal feedback, an intensity of the thermal feedback and atype of the thermal feedback based on at least one of a remaining amountof the health points and a ratio of the remaining amount to a totalamount of the health point, during a reproduction of the game; andcontrolling the feedback device to output the thermal feedback accordingto the determined at least one of the whether or not to output thethermal feedback, the intensity and the type.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute an electronic game including a playeracting according to a manipulation of a user, wherein the player hashealth points and dies in the game when all of the health points isexhausted, obtain at least one of an intensity of the thermal feedbackand a type of the thermal feedback according to a change of the healthpoints during a reproduction of the game; and control, via thecommunication module, the feedback device to output the thermal feedbackaccording to the determined at least one of the intensity and the type,wherein the controller determines the intensity of the thermal feedbackbased on at least one of a change amount of the health points, anddetermines the type of the thermal feedback based on whether the healthpoints is increased or decreased.

The controller may obtain a first intensity as the intensity of thethermal feedback when the change amount is a first value, and obtain asecond intensity greater than the first intensity as the intensity ofthe thermal feedback when the change amount is a second value greaterthan the first value.

The controller may obtain a first intensity as the intensity of thethermal feedback when the ratio is a first value, and obtain a secondintensity greater than the first intensity as the intensity of thethermal feedback when the ratio is a second value greater than the firstvalue.

The controller may obtain one of a hot feedback and a cold feedback asthe type of the thermal feedback when the health points is increased,and obtain another of the hot feedback and the cold feedback as the typeof the thermal feedback when the health points is decreased.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute an electronic game including a playeracting according to a manipulation of a user, wherein the player hashealth points and dies in the game when all of the health points isexhausted, determine at least one of whether or not to output thethermal feedback, an intensity of the thermal feedback and a type of thethermal feedback based on at least one of a remaining amount of thehealth points and a ratio of the remaining amount to a total amount ofthe health point, during a reproduction of the game, and control, viathe communication module, the feedback device to output the thermalfeedback according to the determined at least one of the whether or notto output the thermal feedback, the intensity and the type.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing an electronicgame including a player acting according to a manipulation of a user,and wherein the player has health points and dies in the game when allof the health points is exhausted, including: a casing having a gripportion gripped by a user and forming an exterior of the gamingcontroller; an input module receiving the user input according to amanipulation of the user; a communication module communicating with thecontent reproduction device; a heat outputting module including athermoelectric element performing a thermoelectric operation, a powerterminal applying a power to the thermoelectric element, and a contactsurface which is disposed on the grip portion and configured to contactwith the user, wherein the heat outputting module outputs the thermalfeedback by transmitting, via the contact surface, a heat generated bythe thermoelectric operation to the user; and a controller configured toreceive, via the communication module, the user input, send, via thecommunication module, the received user input to the contentreproduction device to cause the player acts corresponding to themanipulation of the user, and apply an operating power according to achange of the health points to the power terminal so that the heatoutputting module outputs the thermal feedback corresponding to thechange of the health points, wherein the controller performs at leastone of a first operation, a second operation and a third operation,wherein the controller performs the first operation by applying a firstoperating power when a change amount of the health points or a ratio ofthe change amount to a total amount of the health points is a firstvalue and applying a second operating power of which the voltagemagnitude is greater than that of the first operating power when thechange amount or the ratio is a second value greater than the firstvalue, wherein the controller performs the second operation by applyingone of a forward power for the hot feedback and a reverser power for thecold feedback when the health points is increased and applying anotherof the forward power and the reverser power when the health points isdecreased, and wherein the controller performs the third operation byapplying a third operating power when a remaining amount of the healthpoints or a ratio of the remaining amount to the total amount is a thirdvalue and applying a fourth operating power of which the voltagemagnitude is greater than that of the third operating power when theremaining amount or the ratio of the remaining amount is a fourth valuesmaller than the third value.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicecooperating with a feedback device outputting the thermal feedback usinga thermoelectric element, including: executing a multimedia contentprovided as an electronic game or a feedback application, wherein themultimedia content includes a virtual heat source to which a heattransferring attribute including a conduction type and a radiation typeis assigned; determining, based on the heat transferring attribute ofthe virtual heat source, a virtual heat transferring amount transferredfrom the virtual heat source to a player of the multimedia content,obtaining an intensity of the thermal feedback based on the determinedvirtual heat transferring amount; and controlling the feedback device tooutput the thermal feedback having the determined intensity. wherein thedetermining the virtual heat transferring amount includes calculatingthe virtual heat transferring amount based on a temperature value of thevirtual heat source when the heat transferring attribute of the virtualheat source is the conduction type, and calculating the virtual heattransferring amount based on the temperature value and a distancebetween the player and the virtual heat source when the heattransferring attribute of the virtual heat source is the radiation type.

The determining may include when the player is separated from the heatsource of the conduction type, determining that the virtual heat sourceof the conduction type transfers no virtual heat to the player or thatthe virtual heat transferring amount is zero (0).

In a step of the calculating the virtual heat transferring amountrelated to the virtual heat source of the radiation type, the virtualheat transferring amount may get greater as the distance gets smaller.

In a step of the calculating the virtual heat transferring amountrelated to the virtual heat source of the radiation type, a first heatamount may be obtained as the virtual heat transferring amount when thedistance is a first distance, and a second heat amount greater than thefirst heat amount may be obtained as the virtual heat transferringamount when the distance is a second distance smaller than the firstdistance.

The heat transferring attribute may further include a directional type.The determining may further include calculating the virtual heattransferring amount based on the temperature value of the virtual heatsource when the heat transferring attribute of the virtual heat sourceis the directional type, the virtual heat transferring amount related tothe virtual heat source of the directional type being constant even whenthe distance between the player and the virtual heat source varies.

The heat transferring attribute may further include an area type. Thedetermining may further include calculating the virtual heattransferring amount based on the temperature value of the virtual heatsource when the heat transferring attribute of the virtual heat sourceis the area type. And a virtual heat may transfer only when the distancebetween the player and the virtual heat source of the area type issmaller than a predetermined distance.

The method may further include: determining whether a type of thethermal feedback is a hot feedback or a cold feedback based on whetherthe virtual heat transferring amount is positive or negative. And thecontrolling may include controlling the feedback device to outputs thethermal feedback having the determined type of the thermal feedback.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device cooperateswith a feedback device outputting the thermal feedback using athermoelectric element, including: a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute a multimedia content provided as anelectronic game or a feedback application, wherein the multimediacontent includes a virtual heat source to which a heat transferringattribute including a conduction type and a radiation type is assigned,determine, based on the heat transferring attribute of the virtual heatsource, a virtual heat transferring amount transferred from the virtualheat source to a player of the multimedia content, obtain an intensityof the thermal feedback based on the determined virtual heattransferring amount, and control, via the communication module, thefeedback device to output the thermal feedback having the determinedintensity. wherein the controller calculates the virtual heattransferring amount the virtual heat transferring amount based on atemperature value of the virtual heat source when the heat transferringattribute of the virtual heat source is the conduction type, andcalculates the virtual heat transferring amount based on the temperaturevalue and a distance between the player and the virtual heat source whenthe heat transferring attribute of the virtual heat source is theradiation type.

The controller may determine that the virtual heat source of theconduction type transfers no virtual heat to the player or that thevirtual heat transferring amount is 0 when the player is separated fromthe heat source of the conduction type.

The controller may determine that the virtual heat transferring amountrelated to the virtual heat source of the radiation type gets greater asthe distance gets smaller

The controller may determine that the virtual heat transferring amountis a first heat amount when the distance is a first distance, anddetermine that the virtual heat transferring amount a second heat amountgreater than the first heat amount when the distance is a seconddistance smaller than the first distance.

The heat transferring attribute may further include a directional type.And the controller may calculate the virtual heat transferring amountrelated to the virtual heat source of the directional type based on thetemperature value of the virtual heat source, and may determine thevirtual heat transferring amount related to the virtual heat source ofthe directional type being constant even when the distance between theplayer and the virtual heat source varies.

The heat transferring attribute may further include an area type. Andthe controller may calculate the virtual heat transferring amountrelated to the virtual heat source of the area type based on thetemperature value of the virtual heat source, and determine that avirtual heat transfers only when the distance between the player and thevirtual heat source of the area type is smaller than a predetermineddistance.

The controller determine whether a type of the thermal feedback is a hotfeedback or a cold feedback based on whether the virtual heattransferring amount is positive or negative, and control the feedbackdevice to outputs the thermal feedback having the determined type of thethermal feedback.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting the thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing an electronicgame or a feedback application which includes a virtual heat source, andoutputs the thermal feedback corresponding to a virtual heattransferring amount transferred from the virtual heat source to a playerof the game or the application, including: a heat outputting moduleincluding a thermoelectric element which performs a thermoelectricoperation including a heat generating operation, a heat absorbingoperation and a thermal grill operation in which the heat generatingoperation and the heat absorbing operation, a power terminal applying apower to the thermoelectric element, and a contact surface which isdisposed on the grip portion and configured to contact with the user,wherein the heat outputting module outputs the thermal feedback bytransmitting, via the contact surface, a heat generated by thethermoelectric operation to the user; and a controller configured to:control the thermoelectric element to output the thermal feedbackreflecting the virtual heat transferring amount transferred by aconduction from a first virtual heat source to the player, by applying afirst power to the power terminal when the player is contacted with thefirst virtual heat source and by stopping the application of the firstpower when the player is apart from the first virtual heat source, andcontrol the thermoelectric element to output the thermal feedbackreflecting the virtual heat transferring amount transferred by aradiation from a second virtual heat source of a different type than afirst virtual heat source, by applying a second power to the powerterminal when the player is spaced a first distance from the secondvirtual heat source and by applying a third power greater than thesecond power to the power terminal when the player is spaced a seconddistance smaller than the first distance the from the second virtualheat source.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction devicewhich executes a multimedia content provided as an electronic game or afeedback application and cooperates with a feedback device outputting athermal feedback using a thermoelectric element, the method may include:executing the multimedia content, wherein the multimedia contentincludes a virtual object and a player, and implements an interactionbetween the player and the virtual object such as a touch or a grab, thevirtual object having a thermal attribute including a temperatureinformation and a texture information; changing an intensity of thethermal feedback according to at least one of a lasting duration of theinteraction, the temperature information and the texture information;and controlling the feedback device to output the thermal feedbackhaving the intensity.

The method may further include: setting a maximum intensity of thethermal feedback based on the temperature information, increasing theintensity as the lasting duration increases, and stopping changing theintensity of the thermal feedback when the intensity reaches the maximumintensity.

The changing may include adjusting an intensity change rate per timebased on the texture information.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device executes amultimedia content provided as an electronic game or a feedbackapplication, and cooperates with a feedback device outputting thethermal feedback using a thermoelectric element, including: a heatoutputting module including a thermoelectric element which performs aheat generating operation and a heat absorbing operation a powerterminal applying a power to the thermoelectric element, and a contactsurface which is disposed on the grip portion and configured to contactwith the user, wherein the heat outputting module outputs the thermalfeedback by transmitting, via the contact surface, a heat generated fromthe thermoelectric element to the user; a memory storing a data; acommunication module communicating with an external device; and acontroller configured to: execute the multimedia content, wherein themultimedia content includes a virtual object and a player, andimplements an interaction between the player and the virtual object suchas a touch or a grab, the virtual object having a thermal attributeincluding a temperature information and a texture information, change anintensity of the thermal feedback according to at least one of a lastingduration of the interaction, the temperature information and the textureinformation, control the feedback device to output the thermal feedbackhaving the intensity.

The controller may set a maximum intensity of the thermal feedback basedon the temperature information, increase the intensity as the lastingduration increases, and stop changing the intensity of the thermalfeedback when the intensity reaches the maximum intensity.

The controller may adjust an intensity change rate per time based on thetexture information.

Another aspect of the present disclosure is directed to a feedbackdevice for outputting a thermal feedback, wherein the feedback devicecooperates with a content reproduction device executing a multimediacontent provided as an electronic game or a feedback application, andwherein the multimedia content includes a virtual object and a player,and implements an interaction between the player and the virtual objectsuch as a touch or a grab, the virtual object having a thermal attributeincluding a temperature information and a texture information, thefeedback device may include: a casing having a grip portion gripped by auser and forming an exterior of the feedback device; an input modulereceiving the user input according to a manipulation of the user; acommunication module communicating with the content reproduction device;a heat outputting module including a thermoelectric element performing athermoelectric operation, a power terminal applying a power to thethermoelectric element, and a contact surface which is disposed on thegrip portion and configured to contact with the user, wherein the heatoutputting module outputs the thermal feedback by transmitting, via thecontact surface, a heat generated by the thermoelectric operation to theuser; and a feedback controller configured to increase a voltagemagnitude or a current magnitude of the power applied to thethermoelectric element as a lasting duration of the interactionincreases when the interaction between the player and the virtual objectis started according to the user's operation.

The feedback controller may increase the voltage magnitude or thecurrent magnitude of the power with a first rate when the textureinformation of the virtual object indicates a tree, and increase thevoltage magnitude or the current magnitude of the power with a secondrate greater than the first rate when the texture information of thevirtual object indicates a metal.

The feedback controller may set the maximum voltage or the maximumcurrent to be different when the temperature information of the virtualobject is different.

The feedback controller may set a change rate of the voltage magnitudeor the current magnitude differently when the texture information of thevirtual object is different.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing an electronic gameusing a physic engine supporting a collision processing function relatedto a collision between virtual objects, wherein the collision processingfunction includes a collision type in which a behavior of the virtualobjects is calculated considering at least a momentum of the virtualobjects and a trigger type in which one virtual object passes throughanother virtual object; determining the collision processing functionrelated to a get-hit event which occurs when a player character gets hitby the virtual object is the collision type or the trigger type;deciding an intensity of the thermal feedback based on a result of thedetermination; and controlling, based on the intensity of the thermalfeedback, an intensity of an thermoelectric operation performed by afeedback device, wherein the feedback device outputs the thermalfeedback using a thermoelectric element performing the thermoelectricoperation.

The deciding may include deciding the intensity of the thermal feedbackbased on the momentum calculated using the collision processing functionof the collision type when the get-hit event relates to the collisiontype.

The intensity of the thermal feedback may get greater as the calculatedmomentum gets greater.

The deciding may include deciding the intensity of the thermal feedbackto be a predetermined value when the get-hit event relates to thetrigger type.

The deciding may include deciding the intensity of the thermal feedbackbased on at least one of an identification information of a first objectwhich is the virtual object hitting the player character and anidentification information of a second object which is the virtualobject launching the first object when the get-hit event relates to thetrigger type.

Another aspect of the present disclosure is directed to a contentreproduction device, cooperating with a feedback device outputting athermal feedback using a thermoelectric element, including: a memorystoring a data; a communication module communicating with the feedbackdevice; and a controller configured to: execute an electronic game usinga physic engine supporting a collision processing function related to acollision between virtual objects, wherein the collision processingfunction includes a collision type in which a behavior of the virtualobjects is calculated considering at least a momentum of the virtualobjects and a trigger type in which one virtual object passes throughanother virtual object, determine the collision processing functionrelated to a get-hit event which occurs when a player character gets hitby the virtual object is the collision type or the trigger type, decidean intensity of the thermal feedback based on a result of thedetermination, and control, via the communication module, an intensityof an thermoelectric operation performed by the thermoelectric elementbased on the intensity of the thermal feedback.

The controller may decide the intensity of the thermal feedback based onthe momentum calculated using the collision processing function of thecollision type when the get-hit event relates to the collision type.

The intensity of the thermal feedback may get greater as the calculatedmomentum gets greater.

The controller may decide the intensity of the thermal feedback to be apredetermined value when the get-hit event relates to the trigger type.

The controller may decide the intensity of the thermal feedback based onat least one of an identification information of a first object which isthe virtual object hitting the player character and an identificationinformation of a second object which is the virtual object launching thefirst object when the get-hit event relates to the trigger type.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, performed by a content reproduction deviceexecuting a multimedia content and cooperating with a plurality offeedback devices outputting a thermal feedback using a thermoelectricelement, including: executing a virtual reality application providing avirtual space; obtaining a FOV (Field-Of-View) of the virtual spaceaccording to a direction of a user's sight detected from a HMD(Head-Mounted-Display); when a thermal event occurs in the virtualspace, determining at least one target device among the plurality of thefeedback devices based on an orientation of the thermal event withrespect to the FOV; and transmitting a signal instructing outputting thethermal feedback to the target feedback device.

The determining may include determining that the at least one targetdevice includes a first device grabbed by a right hand of the user and asecond device grabbed by a left hand of the user when the thermal eventis located in a central region of the FOV.

The method may further include: control the first device and the seconddevice to output the thermal feedback with a same intensity.

The determining may include determining that the at least one targetdevice includes a first device grabbed by a right hand of the user whenthe thermal event is located in a right region of the FOV anddetermining that the at least one target device includes a second devicegrabbed by a left hand of the user when the thermal event is located ina left region of the FOV.

The determining may include determining that the at least one targetdevice includes a first device grabbed by a right hand of the user and asecond device grabbed by a left hand of the user when the thermal eventis located in a right region or a left region of the FOV. And the methodmay further include: adjusting an intensity of the thermal feedback ofthe first device and the second device differently.

The intensity of the thermal feedback of one of the first device and thesecond device may be greater than the intensity of the thermal feedbackof another of the first device and the second device.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with a plurality of feedback devicesoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback devices; a controller configured to execute a virtual realityapplication providing a virtual space, obtain a FOV (Field-Of-View) ofthe virtual space according to a direction of a user's sight detectedfrom a HMD (Head-Mounted-Display), when a thermal event occurs in thevirtual space, determine at least one target device among the pluralityof the feedback devices based on an orientation of the thermal eventwith respect to the FOV, and transmit a signal instructing outputtingthe thermal feedback to the target feedback device.

The controller may determine that the at least one target deviceincludes a first device grabbed by a right hand of the user and a seconddevice grabbed by a left hand of the user when the thermal event islocated in a central region of the FOV.

The controller may control the first device and the second device tooutput the thermal feedback with a same intensity.

The controller may determine that the at least one target deviceincludes a first device grabbed by a right hand of the user when thethermal event is located in a right region of the FOV and determinesthat the at least one target device includes a second device grabbed bya left hand of the user when the thermal event is located in a leftregion of the FOV.

The controller may determine that the at least one target deviceincludes a first device grabbed by a right hand of the user and a seconddevice grabbed by a left hand of the user when the thermal event islocated in a right region or a left region of the FOV, and adjusts anintensity of the thermal feedback of the first device and the seconddevice differently.

The controller may determine that the intensity of the thermal feedbackof one of the first device and the second device is greater than theintensity of the thermal feedback of another of the first device and thesecond device.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned anda virtual object to which the temperature attribute is assigned; when anarea event reflecting that a player character enters the virtual areaoccurs, control a feedback device to output the thermal feedback relatedto the area event, wherein the feedback device outputs the thermalfeedback using a thermoelectric element performing a thermoelectricoperation; detecting an occurrence of an object event reflecting that aplayer is influenced by the virtual object when the area event islasting; and when the occurrence of the object event is detected duringthe area event, controlling the feedback device to override the thermalfeedback related to the area event by the thermal feedback related tothe object event.

The method may further include: detecting an occurrence of a new areaevent when the object event is lasting; and when the occurrence of thenew area event is detected during the object event, controlling thefeedback device to override the thermal feedback related to the objectevent by the thermal feedback related to the new area event.

The method may further include: when the object event or outputting thethermal feedback related to the object event is finished, controllingthe feedback device to restart outputting the thermal feedback relatedto the area event.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with at least one feedback deviceoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback device; a controller configured to: execute a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned anda virtual object to which the temperature attribute is assigned, when anarea event reflecting that a player character enters the virtual areaoccurs, control, via the communication module, the feedback device tooutput the thermal feedback related to the area event, wherein thefeedback device outputs the thermal feedback using a thermoelectricelement performing a thermoelectric operation, detect an occurrence ofan object event reflecting that a player is influenced by the virtualobject when the area event is lasting, and when the occurrence of theobject event is detected during the area event, control, via thecommunication module, the feedback device to override the thermalfeedback related to the area event by the thermal feedback related tothe object event.

The controller may detect an occurrence of a new area event when theobject event is lasting; and when the occurrence of the new area eventis detected during the object event, control, via the communicationmodule, the feedback device to override the thermal feedback related tothe object event by the thermal feedback related to the new area event.

When the object event or outputting the thermal feedback related to theobject event is finished, the controller may control, via thecommunication module, the feedback device to restart outputting thethermal feedback related to the area event.

Another aspect of the present disclosure is directed to a method forproviding a thermal feedback, including: executing a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned,the virtual area having a global area and a local area included in theglobal area; when a player character enters the global area, controllinga feedback device to output the thermal feedback corresponding to thetemperature attribute of the global area, wherein the feedback deviceoutputs the thermal feedback using a thermoelectric element performing athermoelectric operation; and when the player character enters the localarea, controlling the feedback device to override the thermal feedbackcorresponding to the temperature attribute of the global area with thethermal feedback corresponding to the temperature attribute of the localarea.

The method may further include: when the player character departs thelocal area, controlling the feedback device to restart outputting thethermal feedback corresponding to the temperature attribute of the localarea.

Another aspect of the present disclosure is directed to a contentreproduction device, wherein the content reproduction device reproducesa multimedia content and cooperates with at least one feedback deviceoutputting a thermal feedback using a thermoelectric element, including:a memory storing a data; a communication module communicating with thefeedback device; a controller configured to: execute a virtual realityapplication providing a virtual space, wherein the virtual spaceincludes a virtual area to which a temperature attribute is assigned,the virtual area having a global area and a local area included in theglobal area, when a player character enters the global area, control,via the communication module, a feedback device to output the thermalfeedback corresponding to the temperature attribute of the global area,wherein the feedback device outputs the thermal feedback using athermoelectric element performing a thermoelectric operation, and whenthe player character enters the local area, control, via thecommunication module, the feedback device to override the thermalfeedback corresponding to the temperature attribute of the global areawith the thermal feedback corresponding to the temperature attribute ofthe local area.

When the player character departs the local area, the controller maycontrol, via the communication module, the feedback device to restartoutputting the thermal feedback corresponding to the temperatureattribute of the local area.

1. Thermal Feedback Providing System

Hereinafter, a thermal feedback providing system 1000 according to anembodiment of the present disclosure will be described.

1.1. Overview of Thermal Feedback System

The thermal feedback providing system 1000 according to an embodiment ofthe present disclosure is a system that allows a user to experienceThermal Experiences (TX). Specifically, the thermal feedback providingsystem 1000 may allow the user to experience a thermal experience byoutputting thermal feedback as part of representing multimedia content.

The thermal feedback is a kind of thermal stimulation that makes theuser feel a thermal sensation by stimulating the thermal sensory organsof the user, which are distributed throughout the body of the user. Inthe present specification, thermal feedback refers to all the thermalstimuli that may stimulate the user's thermal sensory system.

Representative examples of the thermal feedback include a hot feedbackand a cold feedback. The hot feedback means the thermal feedback makingthe user feel a hot sensation by applying a “hot heat” or a positiveheat to a hot spot on the user's skin. The cold feedback means thethermal feedback making the user feel a cold sensation by applying a“cold heat” or a negative heat to a cold spot on the user's skin.

Since the heat is a physical quantity represented by a scalar form, theexpression, “apply cold heat,” “or “apply negative heat,” may not be anexact expression from a physical point of view. For the convenience ofthe present description, however, “absorbing heat” may be referred to“applying cold heat” or “transferring cold heat.” The term of “negativeheat” may be also used instead of “cold heat.”

The thermal feedback in the present specification may further include athermal grill feedback in addition to the hot feedback and the coldfeedback. When the hot heat and the cold heat are given at the sametime, the user perceives a pain sensation instead of recognizing the hotsensation and the cold sensation individually. This pain sensation isreferred to as a so-called thermal grill illusion (TGI). That is, thethermal grill feedback means a thermal feedback which applies acombination of the hot heat and the cold heat, and can be provided byoutputting the hot feedback and the cold feedback simultaneously. A moredetailed explanation of the thermal grill feedback will be providedbelow.

The multimedia content may include various kinds of content including amoving picture, a game, a virtual reality application, an augmentedreality application, and the like.

In general, the multimedia content are provided to the user mainly inaccordance with representing audiovisual information. In embodiments ofthe present disclosure, however, the thermal experience based on theabove-mentioned thermal feedback can be included as part of themultimedia content.

Furthermore, the “playback” or “reproduction” of multimedia contentshould be interpreted to include all operations of executing multimediacontent and representing it to users. Therefore, the term “playback” inthe present specification should be construed to include not only anoperation of reproducing a moving picture through a media player butalso an operation of executing a game program, a training program, avirtual reality application, an augmented reality application, etc.

1.2. Configuration of a Thermal Feedback System

FIG. 1 is a block diagram of a configuration of a thermal feedbackproviding system 1000 according to an embodiment of the presentdisclosure.

Referring to FIG. 1, a thermal feedback providing system 1000 mayinclude a content reproduction device 1200, an audiovisual device 1400,and a feedback device 1600.

The content reproduction device 1200 reproduces the multimedia content,and the audiovisual device 1400 outputs an image and/or an audioaccording to the reproduction of the content, and the feedback device1600 may output thermal feedback in accordance with the multimediacontent.

For example, the content reproduction device 1200 may decode multimediacontent including video data, audio data and thermal feedback data togenerate a video signal, audio signal and a signal relating to thermalfeedback (thermal feedback signal). The video signal and audio signalmay be transmitted to the audiovisual device 1400 and the thermalfeedback signal may be transmitted to the feedback device 1600. Theaudiovisual device 1400 receives the video signal and the audio signaland outputs the video and audio, and the feedback device 1600 receivesthe thermal feedback signal and outputs the thermal feedback.

In some embodiments, the thermal feedback signal may include a thermalfeedback message. For example, content reproduction device 1200 may codea thermal feedback message in the form of one or multiple feedbackmessage signals. For example, using Pulse Digital Modulation (PDM),Phase-Shift Keying (PSK), and/or Quadrature Amplitude Modulation (QAM),content reproduction device may include a message for audiovisual device1400 or feedback device 1600 associated with a thermal feedbackoperation.

Hereinafter, each component of the thermal feedback providing system1000 will be described in more detail.

1.2.1. The Content Reproduction Device

The content reproduction device 1200 reproduces the multimedia content.

FIG. 2 is a block diagram of a configuration of a content reproductiondevice 1200 according to an embodiment of the present disclosure.

Referring to FIG. 2, the content reproduction device 1200 may include acommunication module 1220, a memory 1240, and a controller 1260.

The communication module 1220 may communicate with an external device.The content reproduction device 1200 may transmit and receive data toand from the audiovisual device 1400 and the feedback device 1600through the communication module 1220. For example, the contentreproduction device 1200 may transmit an A/V signal to the audiovisualdevice 1400 via the communication module 1220 or a thermal feedbacksignal to the feedback device 1600. In addition, the contentreproduction device 1200 may access the Internet through thecommunication module 1220 to download the multimedia content.

The communication module 1220 may include a wired-type communicationmodule and a wireless-type communication module. Since the wired-typeand the wireless-type each have advantages and disadvantages, thecontent reproduction device 1200 may be provided with both a wired-typecommunication module and a wireless-type communication module.

In some embodiments, communication module 1220 may be one or moredevices for establishing communication between controller 1260 and otherdevices of feedback providing system 1000 via a network. For example,communication module 1220 may include circuitry and one or more antennasfor communicating wirelessly with memory 1240 using a shortrange/near-field wireless communication protocol such as Bluetooth™,Bluetooth™ LE, WiFi, WiFi Direct, and Zigbee. Further, communicationmodule 1220 may communicate with feedback device 1600 using any knownnetwork protocol including any form of wired or wireless access. LAN(Local Area Network) and USB (Universal Serial Bus) communication aretypical examples of the wire-type communication method, and othermethods may be used. In the case of the wireless-type communicationmethod, a wireless personal area network (WPAN) based communicationmethod such as Bluetooth or Zigbee may be used. The wirelesscommunication protocol is not limited thereto. For example, a WLAN(Wireless Local Area Network) based communication method such as Wi-Fior other known communication methods may be used. Proprietary protocolsdeveloped for game machines or consoles may also be used as awire/wireless communication protocol.

The memory 1240 may store various kinds of information. The memory 1240may store data temporarily or semi-permanently. Examples of the memory1240 include a hard disk drive (HDD), a solid state drive (SSD), a flashmemory, a ROM (Read-Only Memory), and a RAM (Random Access Memory). Thememory 1240 may be provided in a form embedded in the contentreproduction device 1200 or in a detachable form.

The memory 1240 stores various data for the operation of the contentreproduction device 1200 including an operating system (OS) foroperating the content reproduction device 1200 or a content to bereproduced by the content reproduction device 1200.

The controller 1260 may control the overall operation of the contentreproduction device 1200. For example, the controller 1260 may load themultimedia content from the memory 1240 and reproduce it, or maygenerate the video signal, the audio signal, or the thermal feedbacksignal in accordance with the content.

The controller 1260 may be implemented as a CPU (Central ProcessingUnit) or the like in accordance with hardware, software, or acombination thereof. It may be provided in the form of an electroniccircuit that performs a control function by processing an electricalsignal in hardware, and may be provided in a form of a program or a codefor driving a hardware circuit in software.

1.2.2. Audiovisual Device

The audiovisual device 1400 can output video and audio to reproduce themultimedia content.

FIG. 3 is a block diagram of a configuration of an audiovisual device1400 according to an embodiment of the present disclosure.

Referring to FIG. 3, the audiovisual device 1400 may include acommunication module 1420 and an A/V module 1440.

The communication module 1420 may communicate with an external device.The audiovisual device 1400 may transmit and receive data to and fromthe content reproduction device 1200 via the communication module 1420.For example, the audiovisual device 1400 may receive the audio signaland/or video signal from the content reproduction device 1200 via thecommunication module 1420.

The communication module 1420 of the audiovisual device 1400 may beconfigured similar to the communication module 1220 of the contentreproduction device 1200, and a detailed description thereof will beomitted.

The A/V module 1440 may provide video and/or audio content to the user.For this, the A/V module 1440 may include a video module 1442 and anaudio module 1444.

The image module 1442 is generally implemented using a display that canoutput a video according to the video signal received from the contentreproduction device 1200. The audio module 1444 is generally implementedusing a speaker that can output audio according to the audio signalreceived from the content reproduction device 1200.

1.2.3. Feedback Device

The feedback device 1600 may output thermal feedback in response to thereproduction of the multimedia content.

FIG. 4 is a block diagram of a feedback device 1600 according to anembodiment of the present disclosure.

FIG. 4, the feedback device 1600 may include a communication module 1620and a heat output module 1640.

The communication module 1620 may communicate with an external device.The feedback device 1600 may transmit and receive data to and from thecontent reproduction device 1200 via the communication module 1620. Forexample, the feedback device 1600 may receive a thermal feedback signalfrom the content reproduction device 1200 via the communication module1620.

The heat output module 1640 may output thermal feedback. Thermalfeedback may be provided by applying hot heat (positive heat) or coldheat (negative heat) to a user's body. The hot heat (positive heat)and/or the cold heat (negative heat) can be generated by a power outputmodule 1640 and the power output module 1640 may include a contactsurface 1641 contacting the user's body and a thermoelectric elementconnected to the contact surface. The thermal feedback can be providedto the user's body via the contact surface 1641.

The heat output module 1640 may perform a heat generating operation, aheat absorbing operation and/or a thermal grill operation along with athermal feedback signal received from the content reproduction device1200 via the communication module 1620 to output thermal feedback, thena user can experience the output thermal feedback.

A more detailed description of the specific configuration and operationof the heat output module 1640 will be described later.

1.3. Implementations of a Thermal Feedback System

The thermal feedback providing system 1000 having the above-describedconfiguration may be implemented in various forms. Hereinafter, someimplementations of the thermal feedback providing system 1000 will bedescribed.

1.3.1. First Implementation

The first implementation of the thermal feedback providing system 1000-1is related to a system for reproducing an augmented reality applicationor a virtual reality application.

FIG. 5 is a schematic diagram of a first implementation of a thermalfeedback providing system 1000-1 according to an embodiment of thepresent disclosure, and FIG. 6 is a schematic diagram of a firstimplementation of a thermal feedback providing system 1000-1 accordingto an embodiment of the present disclosure

Referring to FIGS. 5 and 6, the thermal feedback providing system 1000-1according to the present embodiment includes a console device 1200-1, ahead-mounted display (HMD) 1400-1, and input device 1600-1. The consoledevice 1200-1 may correspond to the content reproduction device 1200,the HMD 1400-1 may correspond to the audiovisual device 1400, and theinput device 1600-1 may correspond to the feedback device 1600.

Hereinafter, each component of the thermal feedback providing system1000-1 according to this embodiment will be described.

The console device 1200-1 corresponding to the content reproductiondevice 1200 may be provided as an electronic device for reproducing anaugmented reality application or a virtual reality application. Forexample, the console device 1200-1 may include a game console forexecuting a VR application such as a Sony Playstation VR™, or a PCcapable of executing an AR/VR application.

The console device 1200-1 may include a communication module 1220, amemory 1240 and a controller 1260, similar to the above-describedcontent reproduction device 1200.

FIGS. 7 and 8 are schematic diagrams showing an exemplary form of theHMD 1400-1 of the first implementation of the thermal feedback providingsystem 1000-1 according to an embodiment of the present disclosure.

The HMD 1400-1 corresponding to the audiovisual device 1400 may bemounted on the head of the user to provide a video and/or an audio tothe user. The audio may be provided to the user through an earphone orthe like.

For example, as shown in FIG. 7, HMD 1400-1 may be provided as anHMD-type electronic device 1400 a such as the Oculus Rift™ or HTC Vive™and the HMD 1400-1 may be coupled to a PC or game console and may outputvideo through an opaque display.

In another example, the audiovisual device 1400, as shown in FIG. 8, maybe implemented as a glasses-type wearable device 1400 b such as GoogleGlass™. The glasses-type wearable device 1400 b may display a virtualvideo (virtual image) through a transparent display. The user can beprovided, via the transparent display, with the virtual video which canaugment a user's visual experience of the real world. Although theglasses-type wearable device 1400 b is conceptually different from theHMD-type electronic device 1400 a using the conventional opaque display,the HMD 1400-1 according to the present application should be understoodas a comprehensive concept to include the glasses-type wearable device1400 b.

FIG. 9 is a block diagram relating to a configuration of the HMD 1400-1of the first implementation of the thermal feedback providing system1000-1 according to an embodiment of the present disclosure.

Referring to FIG. 9, HMD 1400-1 may include a communication module 1420and an A/V module 1440, similarly to audiovisual device 1400 describedabove.

However, in the present embodiment, the communication module 1420 maytransmit information sensed by the sensing module 1460 to the consoledevice 1200-1 in addition to receiving the A/V signal from the consoledevice 1200-1. Also, the communication module 1420 may be provided as awireless-type so that the HMD 1400-1 mounted on the head of the userdoes not disturb the user's movement.

In addition, the image module 1442 in the A/V module 1440 may includetwo displays. One display is for the user's left eye and the otherdisplay is for the user's right eye, so that the two displays can outputa stereoscopic 3D virtual reality or augmented reality image. The imagemodule 1442 may be provided in the form of a transparent display or aprojector for projecting a virtual image on a transparent glass so thata virtual image and a real image may be viewed by the user together.

Referring back to FIG. 9, the HMD 1400-1 may further include a sensingmodule 1460 and a camera 1480.

The sensing module 1460 may sense various information for realizing anaugmented reality or a virtual reality. In particular, the video outputmay be controlled in accordance with the movement of the head of theuser, so as to provide a realistic augmented reality or virtual realityimage. Therefore, the sensing module 1460 may include a posture sensorfor sensing the user's posture and/or a motion sensor for sensing of theuser's motion.

The camera 1480 may capture an image and/or a video. To realize theaugmented reality, the image photographed by the camera 1480 may beused, so that the user may recognize the actual space. Also, thecaptured image and/or video may be used to generate an augmented imagesynthesized with a virtual image.

FIGS. 10 and 11 are schematic diagrams showing an exemplary form of theinput device 1600-1 of the first implementation of the thermal feedbackproviding system 1000-1 according to an embodiment of the presentdisclosure.

The input device 1600-1 corresponding to the feedback device 1600 mayreceive user input and may output thermal feedback.

For example, as shown in FIG. 10, the input device 1600-1 may beprovided as a bar-type input device 1600 a gripped by the user's handsuch as the gaming controller for the Sony Move Motion™ or Vive™. Thebar-type input device 1600 b may also be implemented in pairs, as shownin FIG. 11, one bar held in each hand of the user.

FIG. 12 is a block diagram relating to a configuration of the inputdevice 1600-1 of the first implementation of the thermal feedbackproviding system 1000-1 according to an embodiment of the presentdisclosure.

Referring to FIG. 12, the input device 1600-1 may include acommunication module 1620 and a heat output module 1640, similarly tothe feedback device 1600 described above.

However, in the present embodiment, the communication module 1620 mayreceive the thermal feedback signal from the console device 1200-1.Also, the communication module 1620 may transmit information related tothe user operation inputted through the input module 1650 and/orinformation sensed by the sensing module 1660 to the console device1200-1. The communication module 1620 may be implemented as awireless-type so that the user can easily maneuver the input module1650.

Referring to FIG. 12, the input device 1600-1 may further include acasing 1610, an input module 1650, a sensing module 1660, and avibration module 1670.

The casing 1610 forms the appearance of the input device 1600-1 and mayhouse other components of the input device 1600-1 therein. Accordingly,the housed components can be protected from an external impact or thelike by the casing.

A grip portion 1612 for gripping the input device 1600-1 by the user maybe provided on the input device 1600-1. The grip portion 1612 is aportion where the input device 1600-1 contacts the user's body. Thetouch surface 1641 of the heat output module 1640 may be provided on thegrip portion 1612. Further, the grip portion 1612 may be made of amaterial having a high frictional force (e.g., rubber or urethane) andmay have a non-slip shape (e.g., irregular shape, etc.). The gripportion may also be made of a material which absorbs perspiration.

Input module 1650 may obtain user input from a user. The input module1650 may be implemented in a button-type and/or a joystick-type, so thatthe user can input the user input by pressing a button and/ormanipulating the joystick in a specific direction. Implementation of theinput module 1650 is not limited to the above-described exemplary forms,however, and any other suitable type of input device (e.g., a wheel, atouch screen, etc.) may be used.

The sensing module 1660 may sense various information related to theinput device 1600-1. Examples of the typical sensing module 1660 includea posture sensor for sensing the posture of the input device 1600-1, amotion sensor for sensing the motion of the input device 1600-1, and abiosensor for sensing a user's biological signal. A gyro sensor and/oran acceleration sensor may be used as the posture sensor and/or themotion sensor. The biosensor may include, e.g., a temperature sensor forsensing a user's body temperature and/or an electrocardiogram sensor forsensing an electrocardiogram.

The vibration module 1670 may output vibration feedback. Vibrationfeedback can serve to further enhance user experience with the gamealong with thermal feedback.

The above-described input device 1600-1 may be provided in various formsother than the bar-type.

FIGS. 13-17 are schematic diagrams illustrating other exemplary forms ofinput device 1600-1 in accordance with embodiments of the presentdisclosure.

Referring to FIG. 13, the input device 1600-1 may be provided as in atwo-handheld gaming controller-type device such as a gaming controllerfor Sony's PlayStation™ Dual Shock™ or Microsoft's Xbox™.

In case that the input device 1600-1 is implemented as the gamingcontroller-type device, two grip portions 1612 may be provided at twospaced apart portions of the casing 1610 so as to be gripped by bothhands. The contact surface 1641 may be provided on each of the gripportions 1612.

In addition, the input device 1600-1 may include a wheel-type device1600 d (used in a racing game) as shown in FIG. 14, a joystick-typedevice 1600 e (used in a flight simulator game) as shown in FIG. 15, agun-type device 1600 f (used in a First Person Shooter (FPS) game) asshown in FIG. 16 and/or a mouse-type device 1600 g (commonly used incomputer gaming environments) as shown in FIG. 17.

Further, in the present embodiment, the thermal feedback providingsystem 1000-1 may further include a wearable device 1600-1′, or theinput device 1600-1 may be replaced with the wearable device 1600-1′. Inthis case, the wearable device 1600-1′ may correspond to the feedbackdevice 1600.

FIGS. 18 and 22 are schematic views showing exemplary forms of thewearable device 1600-1′ according to embodiments of the presentdisclosure.

The wearable device 1600-1′ corresponding to the feedback device 1600may be worn on the user's body to serve as a human-machine interface(HMI) and output thermal feedback.

The wearable device 1600-1′ may be provided in various forms. Forexample, the wearable device 1600-1′ may include a glasses or HMD-typedevice as described above, a suit-type device 1600 h as shown in FIG.18, a glove-type device 1600 i as shown in FIG. 19, a shoe-type device1600 j as shown in FIG. 20, a watch-type device 1600 k as shown in FIG.21, or a band-type device 16001 as shown in FIG. 22.

The wearable device 1600-1′ may include a communication module 1620 anda heat output module 1640, as described above with respect to the inputdevice 1600-1.

The wearable device 1600-1′ may include a body which forms theappearance of the wearable device 1600-1′ and a sensing module 1660 thatcan sense the user's action, posture, and/or biological information. Thecontact surface 1641 of the heat output module 1640 may be formed on thebody of the wearable device.

The wearable device 1600-1′ may also include additional components. Forexample, a band-type or a watch-type wearable device 1600-1′ may have adisplay for displaying various information.

1.3.2. Second Implementation

The second implementation of the thermal feedback providing system1000-2 is related to a system for reproducing an augmented realityapplication or a virtual reality application.

FIG. 23 is a schematic diagram of the second implementation of a thermalfeedback providing system 1000-2 according to an embodiment of thepresent disclosure. FIG. 24 is a block diagram of the secondimplementation of a thermal feedback providing system 1000-2 accordingto an embodiment of the present disclosure.

The second implementation of the thermal feedback providing system1000-2 is for processing an augmented reality application or a virtualreality application and may be provided similarly to the firstimplementation of the thermal feedback providing system 1000-1. However,referring to FIG. 23, the console device 1200-1 may be replaced with thesmart device 1200-2 in the present implementation. The smart device1200-2 may correspond to both the content reproduction device 1200 andthe audiovisual device 1400.

For example, the smart device 1200-2 may be provided in asmartphone-type device such as a Galaxy S8™ of Samsung Electronics. Thesmart device 1200-2 generally includes a display, an audio outputterminal, a camera, a posture sensor, and the like, and may be mountedon the HMD 1400-2 to realize a virtual reality or augmented reality.

FIG. 25 is a block diagram relating to a configuration of the smartdevice 1200-2 of the second implementation of the thermal feedbackproviding system 1000-2 according to the present application.

According to the second implementation, the smart device 1200-2 mayinclude an NV module 1440, a sensing module 1460 and a camera 1480. Inthe second implementation, the A/V module 1440, the sensing module 1460and the camera 1480 may be provided in the smart device instead of theHMD 1400-2.

1.3.3. Third Implementation

The third implementation of the thermal feedback providing system 1000-3is related to a system for reproducing an augmented reality applicationor a virtual reality application.

FIG. 26 is a schematic diagram of the third implementation of a thermalfeedback providing system 1000-3 according to an embodiment of thepresent disclosure. FIG. 27 is a schematic diagram of the thirdimplementation of a thermal feedback providing system 1000-3 accordingto an embodiment of the present disclosure.

The third implementation of the thermal feedback providing system 1000-3is for processing an augmented reality application or a virtual realityapplication and may be provided similarly to the first implementation ofthe thermal feedback providing system 1000-1. However, referring to FIG.27, the console device 1200-1 of the first implementation may beincorporated in the HMD 1200-3 in the third implementation. Therefore,the HMD 1200-3 may correspond to both the content reproduction device1200 and the audiovisual device 1400.

For example, the HMD 1200-3 may be provided in the form of an HMD havinga built-in CPU such as Microsoft's Hololens™. Accordingly, the HMD1200-3 can independently execute the virtual reality or the augmentedreality application without connecting with the console device 1200-1.Accordingly, in the present implementation, the console device 1200-1may be omitted in the thermal feedback providing system 1000-3, and theHMD 1200-3 may include the memory 1240 and the controller 1260.

1.3.4. Fourth Implementation

The fourth embodiment of the thermal feedback providing system 1000-4 isrelated to a system for reproducing multimedia content such as a videocontent or a game content through a conventional 2D screen.

FIG. 28 is a schematic diagram of a fourth implementation of a thermalfeedback providing system 1000-4 according to an embodiment of thepresent disclosure.

The fourth implementation of the thermal feedback providing system1000-4 is for processing multimedia content on a conventional 2D screenand may be provided similarly to the first implementation of the thermalfeedback providing system 1000-1. Referring to FIG. 28, in the presentimplementation, the HMD 1400-1 of the first implementation may bereplaced with a display device 1400-4 that provides a 2D screen. Theconsole device 1200-4 may correspond to the content reproduction device1200, and the display device 1400-4 may correspond to the audiovisualdevice 1400. Also, the input device 1600-4 may correspond to thefeedback device 1600.

The display device 1400-4 may include the communication module 1420 andA/V module 1440 of the first implementation described above. Forexample, the display device 1400-4 may be provided in the form of a TV,a monitor, a projector, or the like. Also, the display device 1400-4 mayprovide a 3D image to the user in a stereoscopic manner.

1.3.5. Fifth Implementation

The fifth implementation of the thermal feedback providing system 1000-5is related to a system for reproducing multimedia content such as avideo content or a game content through a conventional 2D screen.

FIG. 29 is a schematic diagram of a fifth implementation of a thermalfeedback providing system 1000-5 according to an embodiment of thepresent disclosure.

The fifth embodiment of the thermal feedback providing system 1000-5 maybe provided for processing an augmented reality application or a virtualreality application similarly to the fourth implementation of thethermal feedback providing system 1000-4. Referring to FIG. 29, in thepresent implementation, the console device 1200-4 and the display device1400-4 which are described in the fourth implementation may be replacedwith a smart device 1200-5. In the present fifth implementation, theconsole device 1200-4 and the display device 1400-4 may be integrated inthe smart device 1200-5. The smart device 1200-5 may correspond to boththe content reproduction device 1200 and the audiovisual device 1400.

For example, the smart device 1200-5 may be provided in the form of asmart phone, a notebook, a tablet, or the like. Accordingly, in thepresent implementation, the smart device 1200-5 may further include anA/V module 1440.

1.3.6. Sixth Implementation

In the sixth implementation of the thermal feedback providing system1000-6, the thermal feedback providing system 1000-6 may include aportable smart device 1200-6 and a peripheral device 1600-6 connected tothe portable smart device 1200-6. This implementation may be useful to auser when the user is carrying the smart device 1200-6.

FIG. 30 is a schematic diagram of a sixth embodiment of a thermalfeedback providing system 1000-6 according to an embodiment of thepresent disclosure.

In the present implementation, the smart device 1200-6 may be providedsimilarly to the smart device 1000-5 of the fifth implementation.

The peripheral device 1600-6 also includes a basic configuration of thefeedback device 1600 and may additionally include a mount to which theportable smart device 1200-6 can be mounted and a battery for outputtingthermal feedback. For example, the peripheral device 1600-6 may beprovided as a selfie-stick or a smart phone case as shown in FIG. 30.The peripheral device 1600-6 may output thermal feedback in conjunctionwith the portable smart device 1200-6.

2. Heat Output Module

Hereinafter, a heat output module 1640 according to an embodiment of thepresent disclosure will be described.

2.1. Overview of the Heat Output Module

The heat output module 1640 may perform a heat generating operation, aheat absorption operation, and/or a thermal grill operation to outputthermal feedback to the user by applying hot heat and cold heat. In thethermal feedback providing system 1000, the heat output module 1640mounted on the feedback device 1600 outputs thermal feedback to a userof the thermal feedback providing system 1000 when the feedback device1600 receives the thermal feedback signal. Accordingly, the user can beprovided with a thermal feedback.

The heat output module 1640 may use a thermoelectric element such as aPeltier element to perform the heat generating operation, the heatabsorbing operation, or the thermal grill operation.

The Peltier effect is a thermoelectric phenomenon discovered by JeanPeltier in 1834. According to the Peltier effect, when an electriccurrent is made to flow through a junction between two conductors a heatgeneration occurs at the one side of the junction and a heat absorptionoccurs at the other side of the junction. Peltier elements are elementsthat produce such a Peltier effect. Peltier elements were initially madeof a junction of different metals such as Bismuth and Antimony, but inrecent years they have been manufactured by arranging N-P semiconductorsbetween substrates for a higher thermal efficiency.

A Peltier element is capable of generating and absorbing heat on bothsides of the element in substantially instantaneous response withapplication of an electric power, switching between the heat generationand the heat absorption by changing the current direction of the appliedpower, and adjusting an intensity of the heat generation or absorptionprecisely by controlling the magnitude of the voltage or the currentvalue of the applied power. A Peltier element is suitable to be used forthe heat generating operation or heat absorbing operation for thethermal feedback. In particular, with Assignee's development of theflexible thermoelectric element been developed, it is now possible tomanufacture the thermoelectric element in a form that can be easilyplaced in contact with the user's body, and the possibility ofcommercial use as the feedback device 100 is increasing.

The heat outputting module 1640 can perform the heat generatingoperation or the heat absorbing operation as electricity is applied tothe thermoelectric element. Although the heat generation and theabsorption occur at the same time in the thermoelectric elements thatare physically supplied with electric power, in the presentspecification the heat generating operation and the heat absorbingoperation of the heat outputting module 1640 is defined with referenceto the contact surface 1641. More specifically, the heat generatingoperation is an operation that causes heat generation at the contactsurface 1641 in contact with the user's body and the heat absorbingoperation is an operation that causes heat absorption at the contactsurface 1641. For example, the thermoelectric element may bemanufactured by disposing an N-P semiconductor on a substrate such that,when an electric power is applied to the thermoelectric element, heat isgenerated at one side of the thermoelectric element and heat is absorbedat the other side of the thermoelectric element. We may arbitrarilydefine one side of the thermoelectric element facing the body of theuser as the front side, and the opposite side as the rear side. Then anoperation that causes the heat generation at the front side and the heatabsorption at the rear side is defined as the heat generating operation,and an operation that causes the heat absorption at the front side isdefined as the heat absorbing operation.

Since the thermoelectric effect is induced by the electric chargeflowing in the thermoelectric element, it is possible to describe theelectric energy inducing the heat generating operation or the heatabsorbing operation of the heat outputting module 1640 in terms of theelectric current. However, in the present description we will describethe electric energy applied to the thermoelectric element mainly interms of the electric voltage. This is merely for the sake of theconvenience of explanation and a person skilled in the arts wouldunderstand the operation of the disclosed embodiments in terms of theelectric current, based on the voltage-based description. The presentdisclosure is therefore not limited to expression in terms of thevoltage.

2.2. Configuration of the Heat Output Module

FIG. 31 is a block diagram of a configuration of a heat output module1640 according to an embodiment of the present disclosure.

Referring to FIG. 31, the heat output module 1640 includes a contactsurface 1641, a substrate 1642, a thermoelectric couple array 1643disposed on the substrate 1642, a power supply terminal 1640 forapplying power to the heat output module 1640 and a feedback controller1645.

The contact surface 1641 is configured to directly contact the user'sbody to transmit hot heat or cold heat generated by the heat outputmodule 1640 to the user's skin. The portion of the external surface ofthe feedback device 1600 that directly contacts the user's body may bethe contact surface 1641. For example, the contact surface 1641 may beformed on a grip portion of a casing of the feedback device 1600 grippedby the user.

The contact surface 1641 may be configured as a layer directly orindirectly attached to an outer surface (in the direction toward theuser's body) of the thermoelectric couple array 1643 that performs theheat generating operation or the heat absorbing operation in the heatoutput module 1640. This type of contact surface 1641 is disposedbetween the thermoelectric couple array 1643 and the skin of the user toperform heat transfer. The contact surface 1641 may be made of amaterial having a high thermal conductivity so that heat transfer fromthe thermoelectric couple array 1643 to the user's body is performedefficiently. The layer-type contact surface 1641 can prevent directexposure of the thermoelectric couple array 1643 to the outside, therebyprotecting the thermoelectric couple array 1643 from external impacts.

In the above description, the contact surface 1641 is disposed on theouter surface of the thermoelectric couple array 1643. However, theouter surface of the thermoelectric couple array 1643 itself may serveas the contact surface 1641. A part or all of a front surface of thethermoelectric couple array 1643 can be the contact surface 1641.

The substrate 1642 may be configured to support a thermoelectric coupleunit 1645 and may be provided as an insulating material. For example,ceramics may be selected as the material of the substrate 1642. Thesubstrate 1642 may be of a flat plate shape. Alternatively, thesubstrate may have another shape, e.g., to fit the form of the body partintended to receive the thermal feedback.

The substrate 1642 may be provided with a flexible material so it may beformed into various shapes and universally used for the various types offeedback devices 1600. In the feedback controller 1600 of a gamingcontroller-type device, for example, the grip portion where a usergrasps a gaming controller with the palm of a hand may have a curvedsurface. To use the heat output module 1640 with such a curved portionof the body, the substrate 1642 may be made flexible. Examples of theflexible material used for the substrate 1642 include glass fiber andflexible plastic.

Thermoelectric couple array 1643 may include a plurality ofthermoelectric couple units 1645 disposed on a substrate 1642.Semiconductor pairs of N-type and P-type may be used as thethermoelectric couple unit 1241. Alternatively, the thermoelectriccouple unit 1241 may be implemented using different pairs of metals (forexample, Bismuth and Antimony).

In the thermoelectric couple unit 1645, the semiconductor pairs areelectrically connected to each other at one end and electricallyconnected to semiconductor of the adjacent thermoelectric couple unit1645 at the other end. Electrical connection between the semiconductorpair 1645 a and 1645 b or adjacent semiconductor pairs is achieved by aconductor member 1646 disposed on the substrate 1642. The conductormember 1646 may be a lead or an electrode such as copper or silver.

The thermoelectric couple unit 1645 may be electrically connected inseries. The plurality of the thermoelectric couple units 1645 connectedin series may form a thermoelectric couple group 1644. At least one ofthe thermoelectric couple group 1644 may form a thermoelectric couplearray 1643.

The power supply terminal 1647 may apply power to the heat output module1640. The thermoelectric couple array 1643 can generate heat or absorbheat according to the power applied to the power supply terminal 1647.According to a voltage value and/or a current direction of the power, itis determined whether the thermoelectric couple array generates heat orabsorbs heat. A pair of the power supply terminals 1644 may be connectedto one thermoelectric couple group 1644. When the heat output module1640 (the thermoelectric couple array 1643) is configured with aplurality of thermoelectric couple groups 1644, plural pairs of thepower terminals 1644 may be arranged. Accordingly, the voltage value andthe current direction are individually controlled for eachthermoelectric couple group 1643, so as to control whether the heatgenerating operation or the heat absorbing operation is performed foreach individual thermoelectric couple array 1643. Furthermore, a degreeof heat generation or heat absorption also can be controlled for eachindividual thermoelectric couple array 1643 in the heat output module1640.

As will be described later, the power supply terminal 1647 receives theelectrical signal output by the feedback controller 1645. The feedbackcontroller 1645 may adjust the current direction and/or voltage value ofthe electrical signal. Accordingly, the heat generating operation andthe heat absorbing operation of the heat output module 1640 can becontrolled. When a plurality of thermoelectric couple groups 1644 areprovided in the heat output module 1640, the electric signals applied tothe power supply terminals 1644 may be separately controlled for eachthermoelectric couple group 1644.

The feedback controller 1645 may apply an electrical signal to thethermoelectric couple array 1643 via the power supply terminal 1647.Specifically, the feedback controller 1645 may receive informationrelated to thermal feedback from the controller 1260 of the contentreproduction device 1200 via the communication module 1620, and then thefeedback controller 1645 may analyze the received information todetermine a type and degree of the thermoelectric operation. Then, thefeedback controller 1645 may generate an electric signal according tothe determination result. The generated electric signal may be appliedto the power supply terminal 1647 so that the thermoelectric couplearray 1643 can output thermal feedback.

The feedback controller 1645 may perform calculations and processing ofvarious information and control an operation of the heat output module1640 by outputting an electric signal to the heat output moduleaccording to the result of the calculations and processing. Thus, thefeedback controller 1645 may be implemented in a computer or similardevice as a hardware, software or combination thereof. The feedbackcontroller 1645 may be provided in the form of an electronic circuitthat performs a control function by processing an electrical signal. Thefeedback controller 1645 may alternatively be provided in a form of aprogram or a code for driving a microprocessor or other hardwarecircuit.

The feedback device 1600 may also be provided with a plurality of theabove-described thermal output modules 1640. For example, if thefeedback device 1600 has a plurality of the grip portions 1621 as shownin FIG. 13, a thermal output module 1640 may be mounted for each gripportion 1621 of the feedback device 1600.

If a feedback device 1600 is provided with a plurality of thermal outputmodules 1640, the feedback device 1600 may be provided with a feedbackcontroller 1660 for each heat output module 1640. Alternatively, asingle feedback controller 1660 may be provided to integrally manage allof the heat output modules 1640 in the feedback device 1600. Also, asshown in FIG. 11, when a plurality of feedback devices 1600 are providedin the thermal feedback system 1000, one or a plurality of heat outputmodules 1640 may be disposed in each feedback device 1600.

2.3. Type of Heat Output Module

Some embodiments of the heat output module 1640 will be described basedon the description of the configuration of the heat output module 1640.

FIG. 32 is a diagram of one embodiment of a heat output module 1640according to an embodiment of the present disclosure.

Referring to FIG. 32, a pair of substrates 1642 are provided so as toface each other in one embodiment of the heat output module 1640. Acontact surface 1641 is located on an outer surface of one of the twosubstrates 1642 to transmit heat generated by the heat output module1640 to the user's body.

A plurality of thermoelectric couple units 1645 may be disposed betweenthe substrates 1642. Each thermoelectric couple unit 1645 is composed ofa semiconductor pair of an N-type semiconductor and a P-typesemiconductor. In one embodiment of the thermoelectric couple units1645, the N-type semiconductor and the P-type semiconductor areelectrically connected to each other by a conductor member 1646 at oneend. The other ends of the N-type semiconductor and the P-typesemiconductor of the thermoelectric couple unit 1645 are respectivelyconnected to a P-type semiconductor of one adjacent thermoelectriccouple unit and a N-type semiconductor of the other adjacentthermoelectric couple unit 1645. The electrical connection between thethermoelectric couple units is achieved by the substrate 1642.Accordingly, the thermoelectric couple units are connected in series toform one thermoelectric couple group 1644. In this embodiment, since thethermoelectric couple array 1643 is composed of one thermoelectriccouple group 1644 and all the thermoelectric couple units 1645 areconnected in series between the power supply terminals 1644, the thermaloutput module 1640 can perform the same thermoelectric operation overthe entire contact surface (front surface). That is, when power isapplied to the power supply terminals 1644 in one direction, the heatoutputting module 1640 performs the heat generating operation, and whenthe power is applied in the opposite direction, the heat outputtingmodule 1640 performs the heat absorbing operation.

FIG. 33 is a diagram of another embodiment of a heat output module 1640according to an embodiment of the present disclosure.

Referring to FIG. 33, the thermal output module 1640 of anotherembodiment is similar to the one embodiment described above. In thisembodiment, however, the thermoelectric couple array 1643 has aplurality of thermoelectric couple groups 1644 and each thermoelectriccouple group 1644 is connected to the respective power supply terminals1644. For example, a direction of a first current applied to a firstthermoelectric couple group 1644-1 can be different from a direction ofa second current applied to a second thermoelectric couple group 1644-2so that if the first thermoelectric couple group 1644-1 performs theheat generating operation, the second thermoelectric couple group 1644-2may perform the heat absorbing operation. In addition, a first voltagevalue applied to the first thermoelectric couple group 1644-1 can bedifferent from a second voltage value applied to the secondthermoelectric couple group 1644-2 so that a degree of a firstthermoelectric operation performed by the first thermoelectric group1644-1 can be different from a degree of a second thermoelectricoperation performed by the second thermoelectric couple group 1644-2.

As illustrated in FIG. 33, the thermoelectric couple groups 1644 may bearranged in a one-dimensional array in the thermoelectric couple array1643. Alternatively, thermoelectric couple groups 1644 may alternativelybe arranged in a two-dimensional array in the thermoelectric couplearray 1643. FIG. 34 is a diagram of another embodiment of a heat outputmodule 1640 according to an embodiment of the present disclosure. Asshown in FIG. 34, according to one embodiment of the presentapplication, the thermoelectric couple group 1644 may be arranged in atwo-dimensional array in the thermoelectric array 1643. Due to thetwo-dimensional array, the thermoelectric operation of thethermoelectric couple array 1643 may be controlled in two-dimensionalway.

Although the above-described embodiments of the heat output module 1640are described as using a pair of opposed substrates 1642, a singlesubstrate 1642 may also be used. FIG. 35 is another diagram of anotherembodiment of a heat output module 1640 according to an embodiment ofthe present application. Referring to FIG. 35, a thermoelectric coupleunit 1645 may be embedded in a single substrate 1642. In thisembodiment, glass fiber or the like can be used as a material of thesubstrate 1642 and the thermoelectric couple unit 1645 can be supportedat a middle portion thereof by the single substrate 1642. The use of asingle substrate 1642 of the present embodiment can provide greaterflexibility to the heat output module 1640.

Various embodiments of the heat output module 1640 described above canbe combined to each other or modified within the scope of the presentapplication. For example, although the contact surface 1641 is formed onthe front surface of the heat output module 1640 as a separate layerfrom the heat output module 1640 in the above embodiments, one surfaceof the heat output module 1640 can alternatively serve as the contactsurface 1641 without the separate layer. That is, the outer surface ofone substrate 1642 described in the above embodiments can be the contactsurface 1641.

2.4. Thermal Feedback Output

Hereinafter, the thermal feedback output operation performed by thefeedback device 1600 will be described.

The feedback device 1600 may output thermal feedback as the heat outputmodule 1640 performs a heat generating operation or a heat absorbingoperation. The thermal feedback includes a hot feedback, a coldfeedback, and a thermal grill feedback.

The hot feedback can be output by performing the heat generatingoperation, and the cold feedback can be output by performing the heatabsorbing operation. Also, the thermal grill feedback can be outputthrough a thermal grill operation in which the heat generating operationand the heat absorbing operation are simultaneously performed.

Alternatively, the feedback device 1600 can output the above thermalfeedback at various intensities. The intensity of the thermal feedbackcan be adjusted in such a manner that the feedback controller 1645 ofthe heat output module 1640 adjusts a magnitude of the voltage appliedto the thermoelectric couple array 1643 via the power supply terminal1647. Here, a method of controlling the magnitude of the voltageincludes applying a power to the thermoelectric couple array 1643 aftersmoothing a duty signal. That is, a way of adjusting the voltage levelby adjusting the duty rate of the duty signal may be used for adjustingthe magnitude of the voltage (voltage level).

Hereinafter, the heat generating operation, the heat absorbing operationand the thermal grill operation will be described in more detail.

2.4.1. Heat Generating/Absorbing Operation

The feedback device 1600 may perform a heat generating operation byusing the heat output module 1640 to provide the hot feedback to theuser. Similarly, the heat output module 1640 may perform an heatabsorbing operation to provide cold feedback to the user.

FIG. 36 is a diagram illustrating a heat generating operation forproviding hot feedback according to an embodiment of the presentdisclosure, and FIG. 37 is a graph relating to temperature during thehot feedback in accordance with an embodiment of the present disclosure.

Referring to FIG. 36, the heat generating operation may be performed byinducing a heat generating reaction on the contact surface 1641 as thefeedback controller 1645 applies a forward-direction current to thethermoelectric couple array 1643. Here, when the feedback controller1645 applies the forward-direction voltage (hereinafter, the voltagewhich is applied to induce the heat generating reaction on the contactsurface 1641 may be referred to as a “forward-direction voltage” or“forward voltage”) to the thermoelectric couple array 1643, thethermoelectric couple array 1643 starts the heat generating operation.In accordance with the heat generating operation, the temperature of thecontact surface 1641 may rise to a saturation temperature with time asshown in FIG. 37. Therefore, at the beginning of the heat generatingoperation, the user can feel no hot feedback or a weak hot feedback,then the user can start to feel the hot feedback until the temperaturereaches the saturation temperature (hereinafter, referred as “hotsaturation temperature”), and the user may feel consistently a hotfeedback corresponding to the saturation temperature (hot saturationtemperature), after a saturation time (hereinafter, referred to as “hottemperature saturation time” or “hot saturation time”).

FIG. 38 is a diagram illustrating a heat absorbing operation forproviding a cold feedback according to an embodiment of the presentdisclosure, and FIG. 39 is a graph relating to temperature during thecold feedback in accordance with an embodiment of the presentdisclosure.

Referring to FIG. 38, the heat absorbing operation may be performed byinducing a heat absorbing reaction in the contact surface 1641 as thefeedback controller 1645 applies a reverse-direction current to thethermoelectric couple array 1643. Here, when the feedback controller1645 applies the reverse-direction voltage (hereinafter, the voltagewhich is applied to induce the heat absorbing reaction on the contactsurface 1641 may be referred to as a “reverse-direction voltage” or“reverse voltage”) to the thermoelectric couple array 1643, thethermoelectric couple array 1643 starts the heat absorbing operation. Inaccordance with the heat absorbing operation, the temperature of thecontact surface 1641 rises to a saturation temperature with time asshown in FIG. 39. Therefore, at the beginning of the heat absorbingoperation, the user may feel no cold feedback or feel only a weak coldfeedback. Then the user can start to feel the cold feedback until thetemperature reaches the saturation temperature (hereinafter, referred toas “cold saturation temperature”), and the user may feel consistently acold feedback corresponding to the saturation temperature (coldsaturation temperature) after a saturation time (hereinafter, referredto as “cold temperature saturation time” of “cold saturation time”).

Alternatively, when a power is applied to a thermoelectric element, inaddition to a heat generating reaction and a heat absorbing reactionoccurring on both sides of the thermoelectric element, electric energymay be converted into thermal energy. That is, a Joule's heat can begenerated. Therefore, when a voltage of the same magnitude is applied tothe thermoelectric couple array 1643 by changing only the direction ofthe current, the temperature change amount due to the heat generatingoperation may be larger than the temperature change amount due to theheat absorbing operation. The temperature change amount means thetemperature difference between the saturation temperature and an initialtemperature in a state where the heat output module 1640 is notoperated.

Hereinafter, the heat generating operation and the heat absorbingoperation performed by the thermoelectric element using electric energywill be collectively referred to as a “thermoelectric operation”. Inaddition, a thermal grill operation which will be described below canalso be interpreted as a kind of the “thermoelectric operation” sincethe thermal grill operation can be realized by combining the heatgenerating operation and the heat absorbing operation.

2.4.2. Degree of Intensity Control of Heat Generating Operation/HeatAbsorbing Operation

As described above, when the heat output module 1640 performs the heatgenerating operation or the heat absorbing operation, the feedbackcontroller 1645 may control the heat generation degree or the heatabsorption degree of the heat output module 1640 by adjusting amagnitude of the applied voltage. Specifically, the feedback controller1645 may adjust a direction of the current to select the type ofthermoelectric operation among the heat generating operation and theheat absorbing operation, and may adjust the magnitude of the voltage toadjust a degree of the intensity of the hot feedback (which is providedwhen the heat output module 1640 performs the heat generating operation)or the cold feedback (which is provided when the heat output module 1640performs the heat absorbing operation).

FIG. 40 is a graph illustrating the degree of the intensity of thehot/cold feedback based on an adjustment of a magnitude of voltageaccording to an embodiment of the present disclosure.

For example, referring to FIG. 40, the feedback controller 1645 mayapply a voltage with five levels in a forward direction or a backwarddirection so that the feedback device 1600 can provide the user with aten distinguishable thermal feedbacks. However, the number of levelsprovided for the hot and cold feedback need not be the same. Moreover,while FIG. 40 shows the hot feedback and the cold feedback having thesame degree (level) of feedback intensity are implemented by changingthe current direction using the same magnitude voltage, the magnitude ofthe voltage value applied for the same intensity degree (level) of thehot feedback and the cold feedback need not be equal to each other.

FIG. 41 is a graph relating to hot/cold feedback with the sametemperature change amount according to an embodiment of the presentdisclosure.

In general, when the same magnitude of voltage is applied to perform theheat generating operation and the heat absorbing operation, thetemperature change amount of the heat generating operation is largerthan the temperature change amount of the heat absorbing operation.Accordingly, for the same level (degree of intensity) of the hotfeedback and the cold feedback, a magnitude of the voltage for the coldfeedback may be larger than a magnitude of the voltage for the hotfeedback as shown in FIG. 41.

As described above, by adjusting the intensity of the thermal feedback,the thermal feedback may be provided with various levels of intensity,such as strong hot sensation, weak hot sensation, weak cold sensation,strong cold sensation, etc. Such multi-level thermal feedback canprovide a greater immersion for the user in a game environment or avirtual/augmented reality environment, and also makes it possible toinspect a patient's senses more precisely when applied to a medicaldevice.

2.4.3. Thermal Grill Operation 2.4.3.1. Types of Thermal Grill Feedback

The thermal grill feedback may include a neutral thermal grill feedback,a hot thermal grill feedback, and a cold grill feedback.

The neutral thermal grill feedback, the hot thermal grill feedback, andthe cold thermal grill feedback provide the user a neutral thermal grillillusion, a hot thermal grill illusion, and a cold thermal grillillusion, respectively. The neutral thermal grill illusion means athermal grill illusion without feelings of hot and cold. The hot thermalgrill illusion means an illusion that makes a user to feel heat inaddition to the thermal grill illusion. The cold thermal grill illusionmeans an illusion that makes a user feel cold in addition to the thermalgrill illusion.

The neutral thermal grill illusion can be realized when the hot thermalfeedback and the cold thermal feedback are provided simultaneously andprovided within a predetermined ratio range. A ratio for the neutralthermal grill illusion (hereinafter referred to as “neutral ratio’) canbe different for each part of the body that is provided with the thermalgrill feedback, and even if it is the same body part, it may be slightlydifferent for each individual user. In general, a user is inclined tofeel the thermal grill illusion when the body area excited by the hotheat (hot thermal feedback) is larger than the body area excited by thecold heat (cold thermal feedback) or when an amount of hot heat providedto the user is larger than an amount of cold heat provided to the user.

A degree of the intensity of the thermal feedback may be represented asan amount of heat that the feedback device 1600 provides to the bodypart contacting the contact surface 1641, or an amount of heat absorbedfrom the body part. When the thermal feedback is provided to a specificbody part for a specific time period, a degree of the intensity of thethermal feedback can be expressed using a difference between thetemperature (for example, difference between a temperature at initialpoints and a temperature at equilibrium points for the thermal feedback)of the target body part to which the thermal feedback is applied.

Alternatively, human body temperature is usually between 36.5 and 36.9°C., and skin temperature is different from each other person, but it isknown to be about 30˜32° C. on average. Specially, a temperature of thepalm is about 33° C. which is slightly higher than the average skintemperature. The above-mentioned temperature values may be slightlydifferent depending on the individual, and even the same person may varyto some extent according to the current conditions.

According to one experimental example, it was confirmed that the neutralthermal grill illusion was felt when a hot thermal feedback of about 40°C. and a cold thermal feedback of about 20° C. were given to the palm of33° C. In the above experimental example, the hot thermal feedback of+7° C. and the cold thermal feedback of −13° C. are given to the palm of33° C. From the above experimental case, the neutral ratio for a humanpalm can be represented as 1.86 (=|−13|/|7|) in view of temperaturedifference.

As can be understood from this, in most people, when the hot heat andcold heat are continuously applied to the same body part, the neutralratio, which can be represented as a ratio of a hot temperaturedifference (temperature difference between an initial point and at a hotequilibrium point for the hot heat) and a cold temperature difference(temperature difference between an initial point and at a coldequilibrium point for the cold heat), and according to the presentapplication, the neutral ratio may be in a range of 1.5-5.

In addition, the hot thermal grill illusion can be sensed by a user whenthe hot heat is applied over the neutral ratio, and the cold thermalgrill illusion can be sensed by the user when the cold heat is appliedover the neutral ratio.

2.4.3.2. Thermal Grill Operation by an Adjustment of Voltage

The feedback device 1600 may perform a thermal grill operation in avoltage control manner. The thermal grill operation in the voltagecontrol manner can be implemented by the feedback device 1600 in whichthe thermoelectric couple array 1643 includes a plurality ofthermoelectric couple groups 1644. For realizing the thermal grilloperation, the feedback device 1600 has a heat output module 1640including two or more thermoelectric couple groups 1644 that can beindividually controlled.

Specifically, the thermal grill operation in the voltage control mannermay be performed by 1) applying a positive voltage to a part of thethermoelectric couple group 1644 to perform a heat generating operationand 2) applying a reverse voltage to another part of the thermoelectriccouple group 1644 to perform a heat absorbing operation, so that theheat output module 1640 provides a user with the hot thermal feedbackand the cold thermal feedback at the same time.

FIG. 42 is a diagram related to a thermal grill operation according toan embodiment of the present disclosure.

Referring to FIG. 42, a thermoelectric couple array 1643 includes aplurality of thermoelectric couple groups 1644 in a line arrangement.The feedback controller 1645 allows the first thermoelectric couplegroups 1644-1 and 1644-3 (e.g., the thermoelectric couple groups formingthe odd-numbered lines) to perform the heat generating operation andallows the second thermocouple groups 1644-2 and 1644-4 (e.g., thethermoelectric couple groups forming the even-numbered lines) to performthe heat absorbing operation. If the thermoelectric couple groups 1644alternately perform the heat generating operation and the heat absorbingoperation according to the line arrangement, the user can be providedwith the hot thermal feedback and the cold thermal feedback at the sametime. As a result, the thermal grill feedback can be provided to theuser. The division of the thermoelectric couple array 1644 into theodd-numbered lines and the even-numbered lines should be understood asan exemplary case and the scope of the present disclosure should not belimited to above embodiment.

The feedback device 1600 may control the saturation temperature of thefirst thermoelectric couple groups 1644-1 and 1644-3 and the saturationtemperature of the second thermoelectric couple groups 1644-2 and 1644-4to conform to the neutral ratio for the neutral thermal grill feedback.

FIG. 43 is a table of voltages for providing the neutral thermal grillfeedback in a voltage control manner according to an embodiment of thepresent disclosure.

For example, referring to FIG. 43, the feedback controller 1645 mayapply five positive voltages and five negative voltages to the heatoutput module 1640, respectively. The heat output module 1640 maygenerate five heat generating operations and five heat absorbingoperations according to the five positive voltages and the five negativevoltages. A temperature change amount according to a heat generatingoperation having a certain level may be same to a temperature changeamount according to a heat absorbing operation having a same level tothe heat generating operation. However, the relationship betweentemperature change amounts of the same-level heat generating/absorbingoperations is not limited to the above. Assuming that a temperaturechange amount between each adjacent heat generating/absorbing operationsis constant, when the neutral ratio is set to 3, the feedback controller1645 may apply a first-level positive voltage (e.g., minimum-levelpositive voltage) to a first thermoelectric couple group 1644 and athird-level negative voltage to a second thermoelectric couple group1644 such that the heat output module 1640 can provide the neutralthermal grill feedback. When the neutral ratios is set to 2.5, thefeedback controller 1645 may apply a second-level positive voltage tothe first thermoelectric couple group 1644 and a fifth-level negativevoltage to the second thermoelectric couple group 1644. When the neutralratio is set to 2, the feedback controller 1645 may apply a first-levelpositive voltage and a second-level negative voltage of a second-levelpositive voltage and a fourth-level negative voltage to thethermoelectric couple array 1643. A user may feel the thermal grillfeedback which is induced by the combination of the first-level positivevoltage and the second-level negative voltage as more painful than thethermal grill feedback which is provided by the combination of thesecond-level positive voltage and the fourth-level negative voltage.This means that a magnitude of intensity of the latter thermal grillfeedback is larger than one of the former thermal grill feedback. Themagnitude of intensity of the thermal grill feedback can be adjusted bycontrolling the applied voltages. The above description for a method ofproviding the thermal grill feedback should be understood as anexemplary embodiment, thus the present disclosure should not be limitedto the above exemplary embodiment. For example, a number of grades(levels) for the heat generating/absorbing operations is not limited to5, and a number of grades for the heat generating operation need not tobe same a number of grades for the heat absorbing operation.

The feedback controller 1645 may also provide a hot thermal grillfeedback by adjusting the positive voltage and negative voltage to bebelow the neutral ratio, or provide a cold thermal grill feedback byadjusting the positive voltage and the negative voltage to be over theneutral ratio.

For example, referring to FIG. 45, when the feedback controller 1645applies a first-level positive voltage to the first thermoelectriccouple group 1644 and a second-level negative voltage to the secondthermoelectric couple group 1644 (assuming that the neutral ratio is setto 3), the user can experience a hot thermal feedback and a thermalgrill feedback at the same time or a warmer thermal grill feedback thanthe neutral thermal grill feedback. In addition, when the feedbackcontroller 1645 applies a first-level positive voltage and afourth/fifth negative voltage to the heat output module 1640, the usercan be provided with a cooler thermal grill feedback than the neutralthermal grill feedback.

However, in case of providing the hot thermal grill feedback or the coldthermal grill feedback, when the constant voltage and the reversevoltage are applied at a ratio largely deviated from the neutral ratio,there is a problem that the user does not experience the thermal grillillusion. Therefore, the ratio of the positive voltage and the negativevoltage are made adjustable to fall within a proper range.

3. Method of Providing a Thermal Feedback

Hereinafter, a method of providing a thermal feedback according to anembodiment of the present disclosure will be described. The followingdescription will be made with reference to the thermoelectric operationprovided by the heat output module 1640 and with reference to thethermal feedback providing system 1000 according to embodiments of thepresent disclosure. However, this is merely for convenience ofexplanation, and therefore, the method of providing the thermal feedbackaccording to an embodiment of the present disclosure is not limitedthereto.

3.1. Outline of the Method of Providing the Thermal Feedback

FIG. 44 is a basic flowchart of a thermal feedback providing methodaccording to an embodiment of the present disclosure.

Referring to FIG. 44, a method of providing a thermal feedback accordingto an embodiment of the present disclosure includes reproducing amultimedia content by a content reproduction device 1200 (S1),obtaining, by the content reproduction device 1200, a thermal feedbackinformation according to the reproduction of the multimedia content(S2), transmitting, by the content reproduction device 1200, a thermalfeedback signal to a feedback device 1600 according to the thermalfeedback information (S3), and performing, by the feedback device, athermal feedback output operation (S4). Hereinafter, the above-describedsteps will be described specifically.

First, the content reproduction device 1200 may reproduce the multimediacontent (S1).

The multimedia content may be a video, a game, a virtual realityapplication, an augmented reality application, a feedback application,and the like. The controller 1260 of the content reproduction device1200 may load the multimedia content stored in the memory 1240 from thememory 1240. The content reproduction device 1200 may receive themultimedia content through the communication module 1220 and reproducethe multimedia content.

For example, the controller 1260 of the content reproduction device 1200can reproduce multimedia content such as a game or a movie file storedin the memory 1240. For another example, the content reproduction device1200 may receive and reproduce the multimedia content by downloading orstreaming it from the Internet through the communication module 1220.

The content reproduction device 1200 may obtain a thermal feedbackinformation according to the reproduction of the multimedia content(S2).

The multimedia content may include thermal feedback data or algorithmsfor processing thermal feedback. The controller 1260 of the contentreproduction device 1200 may decode the thermal feedback data inaccordance with the reproduction of the multimedia content, or mayperform the thermal feedback processing algorithm to obtain the thermalfeedback information.

The thermal feedback information may include at least one of a thermalfeedback target, a thermal feedback type, a magnitude of intensity forthermal feedback, and time information for thermal feedback.

The thermal feedback target may refer a target which is controlled forproviding the thermal feedback to a user. For example, if a plurality offeedback devices 1600 are used in the thermal feedback providing system1000 (see FIG. 11), or if the feedback device 1600 has a plurality ofheat output modules 1640, or if heat output module 1640 is regionallycontrolled (see FIG. 33), the thermal feedback target may indicate atarget to perform the thermal feedback.

A thermal feedback type may refer a kind of thermal feedback. Forexample, thermal feedback type may include a hot feedback, a coldfeedback, and a thermal grill feedback. The thermal grill feedback mayalso include a neutral thermal grill feedback, a hot thermal grillfeedback, and a cold thermal grill feedback.

The thermal feedback intensity may refer an intensity of the thermalfeedback. In some cases, the thermal feedback intensity may include athermal feedback type. For example, the thermal feedback intensity maybe classified as grades 1 to 10, with grades 1 to 5 being assigned tothe cold thermal feedback and grades 6 to 10 being assigned to the hotthermal feedback. Alternatively, the thermal feedback intensity may beclassified as grades −5 to +5, with negative grades being assigned tothe cold thermal feedback and positive grades being assigned to the hotthermal feedback.

The thermal feedback providing time may refer a time information relatedto output of the thermal feedback. The thermal feedback providing timemay include a start time, an end time and a running time (time duration)of the thermal feedback output.

The content reproduction device 1200 may transmit the thermal feedbacksignal to the feedback device 1600 according to the thermal feedbackinformation (S3), and the feedback device 1600 may receive the thermalfeedback signal and perform a thermal feedback output operation inaccordance with the received signal (S4).

Specifically, the controller 1260 may generate the thermal feedbacksignal based on the thermal feedback information and transmit thethermal feedback signal to the feedback device 1600 via thecommunication module 1220. In a thermal feedback providing system 1000having a plurality of feedback devices 1600, the controller 1260 mayselect a feedback device 1600 to transmit a thermal feedback signalbased on the thermal feedback object information as the thermal feedbacktarget. The feedback controller 1645 may receive the thermal feedbacksignal through the communication module 1620 and perform the thermalfeedback output operation according to the thermal feedback signal.

The thermal feedback signal is a signal for controlling the output ofthe thermal feedback. In some embodiments, the thermal feedback signalmay include a thermal feedback start signal indicating an initiation ofthe thermal feedback output and a thermal feedback end signal indicatinga termination of the thermal feedback output. Additionally oralternatively, the feedback signal may be modulated to transmit athermal feedback message. For example, controller 1260 may perform anamplitude modulation of the signal to transmit a message feedback device1600, which may be then decoded to perform a feedback operation. Othermodulation methods, such as frequency modulation or pulse widthmodulation, may also be used to transmit a message via the thermalfeedback signal.

The controller 1260 of the content reproduction device 1200 may transmitthe start signal (thermal feedback start signal) via the communicationmodule 1220, and then the feedback controller 1645 of the feedbackdevice 1600 may receive the start signal via the communication module1620. When the feedback device 1600 receives the start signal, thefeedback controller 1645 may apply power to the thermoelectric couplearray 1643 in response to the start signal to cause the thermoelectriccouple array 1643 to perform the thermal feedback output operation.

The controller 1260 of the content reproduction device 1200 may transmitthe end signal (thermal feedback end signal) via the communicationmodule 1220, and then the feedback controller 1645 of the feedbackdevice 1600 may receive the end signal via the communication module1620. When the feedback device 1600 receives the end signal, thefeedback controller 1645 may stop applying the power to the thermocouplearray 1643 according to the end signal, so that the thermoelectriccouple array 1643 can stop the thermal feedback output operation.

The thermal feedback end signal may be used as an optional signal.

For example, if the start signal includes the thermal feedback providingtime (especially, end time and/or running time), the feedback controller1645 determines a timing of the thermal feedback output operationaccording to the thermal feedback providing time. For example, thefeedback controller 1645 may initiate the thermal feedback outputoperation according to the start time. For another example, the feedbackcontroller 1645 may terminate the thermal feedback output operationaccording to the end time. For another example, the feedback controller1645 may maintain the thermal feedback output operating according to therunning time (time duration).

In another example, the running time for the thermal feedback outputoperation may set by to be a predetermined time, and the feedbackcontroller 1645 may maintain the thermal feedback output operation forthe predetermined running time. In this case, the end signal may beomitted.

The feedback controller 1645 of the feedback device 1600 may transmitthe thermal feedback report signal to the content reproduction device1200 via the communication module 1620 to report the operating status ofthe heat output module 1640. The feedback device 1600 may transmit thereport signal (thermal feedback report signal) periodically or inresponse to receipt of the thermal feedback signal to the contentreproduction device 1200. The thermal feedback report signal may includeinformation such as whether to output thermal feedback, the type orintensity of the thermal feedback being currently output, thetemperature of the contact surface 1641, the user's bio informationsensed by the sensing module, an error information and/or a batterylevel information.

The thermal feedback output operation of the feedback device 1600 inaccordance with the thermal feedback signal may be accomplished in avariety of ways.

First, the initiation and termination of the thermal feedback outputoperation of the feedback device 1600 may be accomplished as follows.

In one embodiment, the feedback device 1600 may perform the thermalfeedback output operation only while the thermal feedback signal isreceived, and may stop the thermal feedback output operation when thethermal feedback signal is not received any more. In this case, thethermal feedback signal may be transmitted continuously.

In another embodiment, upon receipt of the start signal, the feedbackdevice 1600 may output the thermal feedback during a predetermined time(such as a default running time) or a running time included in the startsignal. After the predetermined time or the running time included in thestart signal is elapsed, the feedback device 1600 may stop the thermalfeedback operation.

In another embodiment, the feedback device 1600 may start the thermalfeedback output operation at the time of reception of the start signal,and stop the thermal feedback output operation at the time of receptionof the end signal.

In some embodiments, the thermal feedback signal may be provided as anON/OFF signal. According to the present application, the thermalfeedback signal, however, may be provided in a form including all or atleast a part of the thermal feedback information described above. Whenthe feedback controller 1645 receives the thermal feedback signal, itmay extract the information contained therein and control the thermalfeedback output operation according to the information. For example, thefeedback controller 1645 may determine which heat output module 1640will perform the thermal feedback output operation based on the thermalfeedback target information. As another example, the feedback controller1645 may determine whether to perform a heat generating operation, aheat absorbing operation, or a thermal grill operation based on thethermal feedback type information. For another example, the feedbackcontroller 1645 may determine the voltage value to be applied to thethermoelectric couple array 1643 based on the thermal feedback intensityinformation. For another example, the feedback controller 1645 maydetermine an initiation time and/or a termination time of the thermalfeedback output operation based on the thermal feedback providing timeinformation. In some embodiments, at least one of thetype/intensity/time of the thermal feedback described above may be setto the thermal feedback device 1600 by default.

3.2. Application of Thermal Feedbacks Providing Method

Traditionally, content such as games and movies have been experienced inthe form of audiovisual forms presented by video or audio. To improvethe user's immersion into the content, a tactile experience which isrepresented by vibration feedback and an olfactory experience usingsmell may be used to support and enhance the existing audiovisualexperience. In addition, recently, there have been developed solutionssuch as virtual reality or augmented reality for providing morerealistic enhanced user experiences.

The thermal feedback providing system 1000 realizes a thermal reality(TR) by outputting thermal feedback in cooperation with varioussituations provided by the conventional methods described above. In thisway, the thermal feedback providing system 1000 enhances the userexperience.

In this regard, with the thermal feedback providing method describedabove, the feedback device 1600 outputs thermal feedback through thethermal feedback signal generated according to the reproduction of themultimedia content by the content reproduction device 1200.

Accordingly, the thermal feedback providing method may be applied tovarious technical fields in which a user experience is requested.Hereinafter, a thermal feedback providing system 1000 and somerepresentative technical fields in which a thermal feedback providingmethod can be utilized will be schematically described.

3.2.1. Virtual Reality (VR)

Virtual reality is a representative example of the representativetechnical field in which the thermal feedback providing system 1000 canbe used.

Virtual reality means creating a virtual environment or situation sothat the user feels as if they are actually in a virtual space.Generally, virtual reality is implemented based on a three-dimensionalimage (presented using an HMD) that dynamically changes according to auser's field of view.

With the development of smart devices and the release of Samsung's GearVR™, VR related market is expected to become bigger in the future.

The thermal feedback providing system 1000 of the present disclosure maycooperate with such a virtual reality application and add a thermalsensation to an existing visual/auditory/tactile sense.

For example, the thermal feedback providing system 1000 may assign atemperature to a specific virtual object placed in a virtual space, andwhen a user's avatar in a virtual reality touches the object, thethermal feedback providing system 1000 may provide a hot (or cold)feedback to the user.

Likewise, the thermal feedback providing system 1000 may assign anappropriate temperature to a virtual space such as a desert or polarregions, and when a user's avatar in a virtual reality is in the virtualspace, the thermal feedback providing system 1000 may output a hotfeedback or a cold feedback to the user according to the appropriatetemperature assigned to the virtual space.

3.2.2. Augmented Reality (AR)

Augmented reality is also a representative example of the field in whichthe thermal feedback providing system 1000 may be applied.

Augmented reality is a mixed reality in that it combines virtualenvironments with the real world by presenting virtual objects over arepresentation of the real world.

Compared to virtual reality, which immerses a user entirely into avirtual space, augmented reality basically enhances the real world byproviding virtual objects or additional virtual information. Therefore,an HMD which is used for the augmented reality may be a glass-typetransparent display such that a virtual image displayed on thetransparent display can be visually sensed by the user. Accordingly, theuser can visually experience the real world augmented with the virtualimage. Otherwise, when an HMD which is not adopt the transparent displayis used for the augmented reality, the HMD may display a synthesizedimage which is generated by combining the virtual image with an realimage captured by a camera 1480 in real time.

Apple's iPhone™ and other smart devices have limited augmented realitycapabilities, and in recent years, interests about AR has grown inaccordance with the release of Microsoft's Hololens™, a standalone HMDtype device.

The thermal feedback providing system 1000 may provide a thermalsensation that works in conjunction with such an augmented realityapplication to assist in an existing visual/auditory user experience.

For example, the thermal feedback providing system 1000 may provideuseful information to a user by outputting a hot feedback as one of theenhancement elements when a hot object enters the user's field of view.

3.2.3. Game Content

The thermal feedback providing system 1000 may be utilized for gamecontent.

The game content is basically interactive content based on theinteraction between the game situation and the users.

The implementation of the game content may be realized through theabove-described virtual reality or augmented reality techniques as wellas the conventional technique reflecting the user's operation on thegame screen outputted through the conventional TV or monitor. Thethermal feedback providing system 1000 may add thermal feedback as partof improving a user's immersion in a game environment implementedthrough the techniques described above. For example, the thermalfeedback providing system 1000 may output thermal feedback when a user'savatar is shot by a gun or the like in a first-person shooter-type game.

3.2.4. Video Content

Also, the thermal feedback providing system 1000 may be utilized forvideo content and the like. The video content may be based on anaudiovisual presentation, and the thermal feedback providing system 1000may allow the multimedia content to provide thermal feedback to a userby outputting thermal feedback corresponding to a specific scenerepresented by the audiovisual presentation. For example, the thermalfeedback providing system 1000 may output the hot feedback in anexplosion scene and output the cold feedback in a scene showing a wintersea.

Although the various application fields of the thermal feedbackproviding system 1000 have been described above, the application fieldsof the thermal feedback providing system 1000 are not limited to theabove examples. In addition to the above-described technical fields, thethermal feedback providing system 1000 may be utilized for variousmultimedia content including education or learning content or medicalapplications.

Accordingly, in the present disclosure, the thermal feedback providingsystem 1000 should be construed as being applicable to any field inwhich thermal feedback may be provided to improve the user experience.

4. Implementation of Methods for Providing Thermal Feedback

In the above, the thermal feedback providing method may be used toimprove the user experience in various technical fields. An appropriatethermal feedback may be output as the multimedia content is reproducedso as to provide a better thermal experience for the user.

Hereinafter, various embodiments of a method for providing a thermalfeedback capable of improving a user experience for each technologyfield will be described.

4.1. First Implementation

A first implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting thermal feedbackduring reproduction of audio and/or video content.

When associating the thermal feedback with the video or audio duringreproduction of the video content, a timing of the thermal feedback maybe synchronized with a specific image (specific video scene) or aspecific sound (specific audio scene). For example, in the case of theexplosion scene, the sensation of the hot feedback may coincide with thevideo of the explosion.

However, if the feedback controller 1645 applies the power for thethermal feedback output at the time of the specific scene, a timedifference may occur between the output timing of the specific scene andthe thermal feedback. This is because even if power is applied to thethermoelectric couple array 1643, it takes some time to reach thetemperature at which the contact surface 1641 can provide the thermalfeedback to the user. That is, there may be a delay between a first timepoint of the power application and a second time point when the user canfeel the thermal feedback. Accordingly, if the power application timepoint is set to be the same as the output of the specific scene, theuser may experience the thermal feedback after the specific scene haspassed. Hereinafter, the time duration from an initiation time ofthermoelectric operation to a sensation time when the user can sense thethermal feedback will be referred to as the “delay time.”

In this embodiment, the synchronization between the video output (oraudio output) and the thermal feedback output can be established toimprove the user experience.

Hereinafter, the specific scene which is associated with the thermalfeedback for enhancing the user experience will be referred to as athermal event scene. Thermal event scenes typically may include eventswhich involves a heat absorbing or a heat generating in the real worldsuch as an explosion. The thermal event scenes according to the presentdisclosure are not limited to the above, and the thermal event scenesmay include any scenes that can be associated with the thermal feedbackto improve user's immersion into the video content. Likewise, a specificaudio which is associated with the thermal feedback for improving theuser experience will be referred to as a thermal event audio or thermalevent audio scene.

FIG. 45 is a flowchart of a first implementation of a thermal feedbackproviding method according to an embodiment of the present disclosure.

Referring to FIG. 45, a first implementation of the thermal feedbackproviding method includes loading video content including a thermalevent scene and a thermal feedback data related to a thermal feedbackassociated with the thermal event scene (S110), outputting a videoaccording to the video data (S120), obtaining a sensation time at whichthe thermal feedback is to be sensed (S130), calculating an initiationtime of thermoelectric operation for a thermal feedback based on thesensation time and a correction time (S140), transmitting a thermalfeedback start signal at the initiation time (S150), and starting athermoelectric operation for outputting thermal feedback in accordancewith the thermal feedback start signal (S160).

Hereinafter, each step of the above-described embodiment will bedescribed in more detail.

The content reproduction device 1200 may load video data including thethermal event scene and the thermal feedback data related to a thermalfeedback associated with the thermal event scene (S110). Specifically,the controller 1260 may load the video content stored in the memory 1240or receive the video content through the communication module 1220 in adownloading method or a streaming method.

The video content may include the video data and the thermal feedbackdata. The video content may be provided as one file including the videodata and the thermal feedback data, but the video content which isassociated with the thermal feedback may alternatively be providedseparately in a video file including the video data and a thermalfeedback file including the thermal feedback data.

FIG. 46 is a diagram illustrating an example of thermal feedback dataused in the first implementation of the thermal feedback providingmethod according to an embodiment of the present disclosure.

According to one embodiment, the thermal feedback data may be providedin a form similar to the subtitle file used to overlay the subtitles atthe time of video output, loaded with the video file, similar to thatshown in FIG. 46.

Here, a correction time may be defined in the section “<HEADER>”. Thecorrection time may be defined for each type of the thermal feedback tobe output. In addition, the correction time may be defined for eachintensity of the thermal feedback to be output. The correction time maycorrespond to the delay time. For example, in the case of FIG. 46, fivecorrection times from the first level to the fifth level are set for thehot feedback and the cold feedback in the header portion.

Here, a time information (thermal feedback providing time), a type, anintensity, a thermal feedback target and/or the thermoelectric couplearray for the thermal feedback may be provided in the “<BODY>” section.As described above, the time information may include the start time andthe end time for the thermal feedback, and the start time may be definedas a time that may be the same as the output timing of the thermal eventscene associated with the thermal feedback.

The video data includes information about scenes to be output at thetime of reproducing the video content. Also, the thermal event scene maybe included in the scenes to be output.

The content reproduction device 1200 may output a video and/or audioaccording to the video data (S120). For example, the controller 1260 maydecode video data using a video codec and output the video. The videooutput may be performed through an external or internal display.

The content reproduction device 1200 may obtain a time point at whichthe thermal feedback is to be sensed (S130). Specifically, thecontroller 1260 may obtain the time point at which the user shouldexperience thermal feedback from the thermal feedback data. The timepoint of the thermal feedback may be the same as the output time of thespecific scene to be associated with the thermal feedback.

The content reproduction device 1200 may calculate the start time of thethermoelectric operation for the thermal feedback based on the obtainedtime point (sensation time) of the thermal feedback and the correctiontime (S140). Specifically, the controller 1260 may calculate the starttime of the thermoelectric operation for the thermal feedback bysubtracting the correction time from the sensation time (the obtainedtime at S130) of the thermal feedback.

The correction time may be a time interval from the power applicationtime point when the power is applied to the thermoelectric couple array1643 until the temperature at which the contact surface 1641 becomes atemperature at which the user can experience thermal feedback.

The controller 1260 may determine the correction time with reference toa correction time table stored in the memory 1240. Alternatively, whenthe video data includes information on the correction time, thecontroller 1260 may determine the correction time with reference to theinformation included in the video data.

The correction time may be a predetermined value irrespective of thetype and intensity of the thermal feedback. Otherwise, as describedabove, the correction time may vary depending on the type and/or theintensity of the thermal feedback.

In this case, the controller 1260 may determine the correction timebased on at least one of the type of the thermal feedback and theintensity of the thermal feedback.

For example, since the delay time for the hot feedback and the delaytime for the cold feedback may be different from each other, thecorrection time may depend on whether the thermal feedback is the hotfeedback or the cold feedback. Specifically, in the case of the hotfeedback and the cold feedback of the same intensity, the time for thecontact surface to reach the saturation temperature by the heatgenerating operation for the hot feedback may be faster than the timefor the contact surface to reach the saturation temperature by the heatabsorbing operation for the cold feedback. That is, the delay time forthe hot feedback may be shorter than the delay time for the coldfeedback.

In some embodiments, the intensity of the thermal feedback may beclassified into a plurality of grades. In such embodiments, the delaytimes for thermal feedback may dependent on the intensity of the thermalfeedback. For example, the delay time of a stronger intensity may bedifferent than the delay times for thermal feedback of a weakerintensity. A stronger intensity may be require a high saturationtemperature while a weaker intensity may only require a low saturationtemperature. Such difference in saturation temperatures may result indifferent temperature change rates, temperature gradients, and/ortemperature change velocities, impacting the delay times. Accordingly,the controller 1260 may determine the correction time based on theintensity of the thermal feedback.

In some embodiments, controller 1260 may determine that the correctiontime for a stronger intensity may be set shorter than the correctiontime for a weaker intensity to take into account differences intemperature change velocities. For example, because a stronger intensityrequires a high saturation temperature and the temperature changesquickly, a short correction time is required to synchronize when contactsurface 1641 reaches a temperature at which the user can experiencethermal feedback. However, the weaker intensity, with a lower saturationtemperature and the slower temperature change rate, may require longercorrection times. In other embodiments, the correction time for thestronger intensity may be set larger than the correction time for theweaker intensity. For example, when thermoelectric couple array 1643 isconfigured to have uniform temperature gradients or changes, or whenthermoelectric couple array 1643 includes thresholds temperaturechanges, controller 1260 may set longer correction times for thestronger intensity based on an expected temperature change velocity.

Alternatively, since the delay time for the thermal feedback may be aninherent characteristic of the feedback device 1600 outputting thethermal feedback, the controller 1260 may determine the delay time inconsideration of the identification information of the feedback device1600. For this, the controller 1260 may receive and obtain theidentification information of the feedback device 1600 through thecommunication module 1220. Alternatively, the feedback device 1600itself may store information regarding the delay time and/or thecorrection time, and the controller 1260 may set the correction timebased on the delay time and/or the correction time information receivedfrom the feedback device 1600.

The content reproduction device 1200 may transmit a thermal feedbackstart signal at the start time of the thermoelectric operation for thethermal feedback (S150). When the start time of the thermoelectricoperation is determined, the controller 1260 controls the communicationmodule 1220 to transmit the thermal feedback start signal to thefeedback device 1600 at the determined start time. The start time may bea relative value with respect to a playback time of the video content.The start time may be determined based on a playback duration (playbacktimeline) of the video content.

The feedback device 1600 may initiate a thermal feedback outputoperation in accordance with the start signal of the thermal feedback(S160).

FIG. 47 is a diagram of a thermal feedback output operation of a firstimplementation of a thermal feedback providing method according to anembodiment of the present disclosure.

Specifically, the feedback controller 1645 applies power to thethermoelectric couple array 1643 in response to the receipt of the startsignal (which is substantially the same as the start time of thethermoelectric operation). The thermoelectric couple array 1643 mayperform the heat generating operation or the heat absorbing operationfrom the power application time. When the correction time elapses fromthe power application time, the temperature of the contact surface 1641may reach a temperature at which the user can feel the thermal feedback.

Accordingly, the user can feel the thermal feedback at the output timingof the thermal event scene at the playback time of the video content.Under the control of the content reproduction device 1200, the feedbackdevice 1600 may apply power to the thermoelectric element at the starttime of the thermoelectric operation, which may be set at a time pointearlier than the output time point of the specific scene to beassociated with the thermal feedback so that the user may experience thethermal feedback at the output time of the thermal event scene.

In the above description, the synchronization between the video and thethermal feedback has been described with reference to this embodiment.However, the audio and the thermal feedback may be synchronized byreplacing the video with the audio. This can be similarly applied toother embodiments of the thermal feedback providing method to bedescribed later.

According to the above-described embodiment, the audiovisual experienceaccording to the video or audio and the thermal experience according tothe thermal feedback may be provided in harmony, so that the userexperience can be improved.

4.2. Second Implementation

In some games or 4D movies, etc., in conventional multimedia content, avibration feedback is associated to video or audio output to improveuser immersion.

The second implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting the vibrationfeedback associated with the thermal feedback.

FIG. 48 is a flowchart of the second embodiment of the thermal feedbackproviding method according to an embodiment of the present disclosure.

Referring to FIG. 48, the second implementation of the thermal feedbackproviding method may include reproducing multimedia content including avibration event (S210), transmitting a vibration feedback start signalwhen the vibration event occurs (S220), and performing a thermalfeedback output operation in accordance with the vibration feedbackstart signal (S230).

Hereinafter, each step of the above-described embodiment will bedescribed in more detail.

The content reproduction device 1200 may reproduce the multimediacontent including the vibration event (S210). Specifically, thecontroller 1260 may load the multimedia content from the memory 1240 orthrough the communication module 1220. For example, the contentreproduction device 1200 may reproduce a video content or execute a gameapplication, a virtual reality application or an augmented realityapplication.

The multimedia content may include a vibration feedback data in additionto video data and audio data. The content reproduction device 1200 maydetermine whether the vibration event has occurred or not according tothe reproduction of the multimedia content based on the vibrationfeedback data. A vibration event is an event that requires the vibrationfeedback output during the reproduction of the multimedia content. Forexample, in the case of video content, the vibration feedback data maybe set as a vibration event requiring a vibration feedback output for areproduction time point such as a car crash scene point or a gun-shotscene point in video data. In another example, the vibration event mayinclude a character being hit, e.g., by a punch, or a character using acertain skill, e.g., jumping a far distance.

When a vibration event occurs, the content reproduction device 1200 maytransmit a vibration feedback start signal to the feedback device 1600(S220), and the feedback device 1600 may perform a thermal feedbackoutput operation in accordance with the vibration feedback start signal(S230).

The controller 1260 may transmit a vibration feedback start signal tothe feedback device 1600 via the communication module 1220 when it isdetermined that the vibration event has occurred. In addition, thefeedback controller 1645 may receive the vibration feedback start signalthrough the communication module 1620, and may apply a power to thethermoelectric couple array 1643 to perform a thermal feedback outputoperation.

Alternatively, if the feedback device 1600 includes the vibration module1670, the feedback device 1600 may output the vibration feedback alongwith the output of the thermal feedback through the vibration module1670.

The present implementation described above is a method for implementinga thermal feedback in multimedia content in which thermal feedback datais absent, but vibration feedback data is present. Other data includedin the multimedia content may be used to output the thermal feedbackinstead of the vibration feedback data.

As an example, the present implementation may be modified to outputthermal feedback in response to a screen shaking event instead of avibration event. Some games, including FPS, use a screen shakingtechnique to shake a virtual camera in the game engine so as to shakethe video presentation when, e.g., a collision occurs. Accordingly, thecontroller 1260 may reproduce the multimedia content including thescreen shaking event instead of the vibration event. In this case, thecontroller 1260 may transmit a thermal feedback start signal to thefeedback device 1600 through the communication module 1220 when a screenshaking event occurs during the reproduction, so that the feedbackcontroller 1645 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal.

As another example, the present implementation may be modified to outputthermal feedback according to a specific audio instead of a vibrationevent. More specifically, when the controller 1260 reproduces multimediacontent and a specific audio is output during the reproduction, thecontroller 1260 may transmit a thermal feedback start signal to thefeedback device 1600 via the communication module 1220, and the feedbackcontroller 1645 may perform a thermal feedback output operationaccording to the feedback start signal. The specific audio may be, forexample, an audio generated in a collision, an audio generated in anexplosion, a scream, or the like. When the audio output is equal to orhigher than a predetermined volume, or falls within a predeterminedfrequency band, the controller 1260 may determine it to trigger athermal feedback.

According to the present implementation, a thermal feedback may beprovided even when there's no thermal feedback data provided with themultimedia content by using other data associated with the audio orvideo content instead of the thermal feedback data.

4.3. Third Implementation

A third implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackassociated with an element attribute of a skill at the time of actuatingthe skill or being hit during a game.

Here, an element attribute is the attribute given to a skill in thegame. The element attributes of a skill may be defined according to theskill attribute design of the game developer. For example, an elementattribute of the skill may include a fire attribute (or a flameattribute), an ice attribute (or a freeze attribute), a wind attribute,a lightning attribute (or an electric attribute), and the like.

In the present implementation, the game should be understood as acomprehensive concept including stereoscopic 3D games, games usingvirtual reality technique or augmented reality techniques, as well asconventional 2D games. The same is also true of other embodiments of thethermal feedback providing method according to embodiments of thepresent disclosure to be described later.

FIG. 49 is a flowchart of a third embodiment of a method for providing athermal feedback according to an embodiment of the present disclosure.

Referring to FIG. 49, the third implementation of the thermal feedbackproviding method may include executing a game in which skills having aplurality of element attributes are implemented (S310), generating athermal feedback start signal including the thermal feedback typeinformation corresponding to the element attribute of the skill whenoccurring a skill actuation event (skill cast event) or a skill hitevent (S320), transmitting the thermal feedback start signal (S330), andperforming a thermal feedback output operation according to the thermalfeedback start signal, wherein the thermal feedback may includeoutputting a thermal feedback corresponding to the thermal feedback typeinformation (S340).

Hereinafter, each step of the above-described embodiment will bedescribed in more detail.

The content reproduction device 1200 may execute a game in which theskills having a plurality of element attributes are implemented (S310).The controller 1260 may load and execute such a game.

The skill may correspond to an operation such as a spell used by avirtual character in a game and/or a shooting action of a projectile,and the like. Another representative example of the skill may includefirewalls, ice bolts, and lightning chains, etc. As some other examples,there may be an action for wielding a weapon, an action for launching aprojectile, etc. In this case, the attribute given to the weapon or theprojectile may be treated as an attribute of the skill. The virtualcharacter may include a player character controlled by the player or anenemy character attacking the player character. In the case of a firstperson game including VR/AR, a field of view of the player charactercontrolled by the player may be displayed and/or shown instead ofdisplaying the player character on the screen in the virtual realityenvironment or the augmented reality environment. Here, the term“player” is a generic term encompassing a virtual character (playablecharacter) controlled by the player (a user) or an avatar thereof in thevirtual space or augmented space.

Skills in the game may have element attributes. For example, there maybe given a fire attribute (flame attribute) in the case of a fire ball,an ice attribute (freeze attribute) in the case of an ice bolt, and, andan electric attribute in the case of a lightning chain.

In addition, each skill may be given a skill level. In an example, askill in a game may be implemented in such a way that a power of theskill increases as the player rises in skill level. For example, afireball skill is given a level, and the fireball of level 1 may beupgraded to a fireball of level 2, in which case an effect of thefireball skill is enhanced.

For skills having a same element attribute, one skill may be superior toanother skill. That is, a skill may be given a skill tier. For example,when skills of the flame attribute include fire bolts, fireballs, andfirestorms in the game, level 1 may be assigned to the fire bolts, level2 may be assigned to the fireballs, and level 3 may be assigned to thefirestorms.

Or, each skill may have a damage value. For example, a fire bolt'sdamage may be given as a value between 100 to 200 damage points, afireball's damage may be given as a value between 200 to 300 damagepoints, and a firestorm's damage may be given as a value between 300 to400 damage points.

The content reproduction device 1200 may generate a thermal feedbackstart signal including thermal feedback type information correspondingto an element attribute of the skill in the case where a skill actuationevent or a shooting event occurs during the game (S320).

When a user input instructing an actuation of a skill in the game isdetected, the controller 1260 may regard the user input as a skillactuation event (skill cast event). Alternatively, when the skillinvoked by another character hits the player character in the game, thecontroller 1260 may determine the hit action as a skill hit event.

When the controller 1260 detects the skill actuation event or the skillhit event, the controller 1260 may determine a type of thermal feedbackbased on an element attribute corresponding to a skill activated by auser player or a skill hitting the user player. For example, the type ofthe thermal feedback may be determined as the hot feedback for a skillhaving a flame attribute, and the type of the thermal feedback may bedetermined as the cold feedback for a skill having an ice attribute. Inaddition, the type of the thermal feedback may be set as the thermalgrill feedback for a skill having an electric attribute.

The controller 1260 may determine an intensity of the thermal feedbackbased on at least one of a skill type, a skill grade (skill level), askill damage (damage of the skill activated by the user player), and thehit damage (damage of the skill hitting on the user player). Forexample, if fire bolts, fireballs and firestorms are implemented in thegame, the intensity for the thermal feedback may be determined as a weakintensity for the fire bolts, as an intermediate intensity for thefireballs, and as a strong intensity for the firestorms. For anotherexample, the intensity of the thermal feedback may be determined as aweak intensity for a fireball of a first level, as an intermediateintensity for a fire ball of a second level, and as a strong intensityfor a fireball of a third level. For another example, the strongerintensity may be determined based on the damage value of the skill. Foranother example, when the skill hit event occurs, the intensity of thethermal feedback may be set to be stronger as the damage on the playerincreases.

The controller 1260 may also determine a duration time for the thermalfeedback. For example, the user player (a virtual character) hit by askill of the flame attribute may suffer a flaming effect for apredetermined time period, and the virtual character hit by a skill ofthe ice attribute may suffer a stop effect or slow effect for apredetermined time period. Also, the character shot by a skill oflightning attribute may suffer a paralyzing effect or the like for apredetermined time period. The controller 1260 may determine a durationtime for providing the thermal feedback according to the duration (thepredetermined time period) of the special effects caused by the skillsfor the character.

These special effects may be a buff effect which is beneficial on thetarget of the skill or a debuff effect which is detrimental on thetarget of the skill. For example, if a debuff effect is generated forthe player according to the skill hit event, the thermal feedback may beoutput during a duration time of the debuff effect. Also, the thermalfeedback may be output during a duration time of the buff effect inducedon a player according to the skill actuation event.

Also, the controller 1260 may gradually decrease the intensity of thethermal feedback during the duration time of the thermal feedback, sothat the user can sense the end points of the buff effect or debuffeffect.

The controller 1260 may generate a thermal feedback start signalincluding information on the type of the thermal feedback. The thermalfeedback start signal may also include information on the intensity ofthe thermal feedback. Also, the thermal feedback start signal may alsoinclude information on the duration time of the thermal feedback.

Alternatively, when the controller 1260 determines the type and theintensity of the thermal feedback, the controller 1260 may refer to askill-thermal feedback table stored in game content or the memory 1240,as shown in FIG. 50.

FIG. 50 is a diagram of a skill-thermal feedback table used in the thirdimplementation of the thermal feedback providing method according to anembodiment of the present disclosure.

Referring to FIG. 50, the table may include information regarding anidentifier of the skill, an element attribute of the skill, a type ofthe thermal feedback for the skill according to the element attribute, atier of the skill, a level of skill, an intensity of the thermalfeedback for the skill, and a time duration for providing the thermalfeedback for the skill.

The content reproduction device 1200 may transmit the thermal feedbackstart signal (S330). The controller 1260 may transmit the generatedthermal feedback start signal to the feedback device 1600 through thecommunication module 1220.

The feedback device 1600 may perform a thermal feedback output operationaccording to the thermal feedback start signal, wherein the thermalfeedback output operation may reflect the thermal feedback typeinformation (S340).

When the thermal feedback start signal is received, the feedbackcontroller 1645 may apply a power to the thermoelectric couple array1643 to perform the thermal feedback output operation. The feedbackcontroller 1645 may generate a power supplying signal according to theinformation included in the thermal feedback start signal. For example,the controller 1260 may determine whether to perform a heat generatingoperation, a heat absorbing operation, or a thermal grill operation byreferring to the type of the thermal feedback, and generate a powersupply signal corresponding thereto. Also, the controller 1260 maydetermine the voltage level by referring to the intensity of the thermalfeedback.

The controller 1260 may also determine a length of time to apply a powersupplying signal based on the providing time or the duration time of thethermal feedback. The controller 1260 may output a thermal feedback of apredetermined constant intensity during the duration time.Alternatively, the thermal feedback may be output with an intensity thatgradually decreases over the time duration. For example, if a virtualcharacter is hit by a skill for the first time, it may output astrong-intensity thermal feedback, and after a predetermined timeperiod, a weaker intensity of the thermal feedback may be output duringthe duration time of a special effect induced by the skill hit event ofthe skill actuation event (for example, paralysis or slowness).According to this, the user can sense whether or not the virtualcharacter was hit by the strong thermal feedback at the hitting time.The user can also sense whether the virtual character is under thespecial effects via the weak thermal feedback. In addition, the user canalso sense whether the special effects have terminated by thetermination of the thermal feedback. This may improve the intuitivenessof the game and the immersion of the user in the game environment.

According to the above-described method, the feedback device 1600 mayapply a constant voltage to the thermoelectric element at the time ofactuating or being hit by the flame attribute skill, and apply a reversevoltage to the thermoelectric element for the cold attribute skill. Thefeedback device 1600 may also control the content reproduction device1200 to apply the constant voltage and the reverse voltage incombination for the lightening attribute skill.

The following description will be made on the basis that the elementattribute of the skill includes fire, ice, or lightning. It should benoted, however, that this is just for convenience of explanation, andthat the elemental attributes of the skill may be defined differently.

In the above description, the hot/cold/thermal grill feedbackcorresponds to the element attribute, but the matching relationshipbetween the type of the thermal feedback and the type of the elementattribute may be freely changed according to the designer's choice. Inthe thermal grill feedback, a neutral thermal grill feedback/hot thermalgrill feedback/cold thermal grill feedback may be used. For example, thehot thermal grill feedback may be matched to the flame attribute, thecold thermal grill feedback to the ice attribute, and the neutralthermal grill feedback to the lightning attribute.

Although the above explanation is mainly focused on the actuation of askill and hits by a skill, a similar thermal feedback may be providedfor other attack actions. For example, the attack action may include aweapon attack in addition to a skill attack. The weapon attacks mayinclude, e.g., melee weapon attacks and ranged weapon attacks.

The element attribute of the attack action may be determined accordingto an element attribute of a weapon used for the weapon attacks insteadof the element attribute of the skill attack. In addition, if the rangedweapon is used for the weapon attack, the element attribute for theweapon attack may be determined according to an element attribute givento a projectile of the ranged weapon. If the element attributes aregiven to both the ranged weapon and the projectile, one of the twoelemental attributes may be prioritized to determine as the elementattribute of the weapon attack.

If the attack action is a skill attack, the intensity of the thermalfeedback may be determined based on at least one of the skill level, thedamage amount of the skill and the skill's tier on a skill treeincluding the plurality of skills having the same element attribute. Ifthe attack action is a weapon attack, the intensity of the thermalfeedback may be determined based on at least one of a class (grade) ofthe weapon, an attack power of the weapon, a class of a projectile ofthe weapon (in case of the ranged weapon) and the attack power of theprojectile.

In addition, the intensity of the thermal feedback may be determinedbased on at least one of an attack power of the attack action, a damageamount of the player according to the skill hit event, a ratio of damageto the total health points of the player, and the remaining healthpoints of the player.

In some embodiments, a type of the thermal feedback may be selectedaccording to a type of the weapon. For example, if the weapon used forthe skill actuation event or the skill hit event is an ice attributeweapon, a cold thermal grill feedback may be matched thereto and if theweapon is a flame attribute weapon, the hot thermal grill feedback maybe matched thereto.

According to the above-described implementation, the user's enjoymentand immersion for the game may be improved by outputting the thermalfeedback corresponding to the element attribute of the skill in responseto the actuation of the skill or the hit by the skill in the game.

4.4. Fourth Implementation

The fourth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting the thermalfeedback associated with an emotional expression or a text selection fora character and/or player.

The emotional expression of the character may include pleasure, anger,fear, and the like. In some cases, an emotional state of the charactermay be defined as a kind of character attribute in a game. In othercases, the character may directly express his emotional state byperforming a certain action or making a certain facial expression. Inthe present implementation, the emotional expression should beinterpreted to include specific actions or facial expression with whicha character expresses his emotions, as well as the attribute directlyindicating the emotion.

Here, a text selection means selecting one among plural selectable textsprovided to the user during the progress of the game. For example, theselectable texts may be presented in the game conversation situationbetween a playable character (PC) operated by a user and a non-playablecharacter (NPC) or in a situation in which a user must make a decision,which the user may effect by selecting one of the texts.

FIG. 51 is a flowchart of the fourth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 51, the fourth implementation of the thermal feedbackproviding method may include executing a game including a virtualcharacter performing emotional expression or a game providing at leastone text being selectable by a user (S410), generating a thermalfeedback start signal including a thermal feedback type informationcorresponding to the emotional expression or the text selection whensensing an emotional expression event or a text providing event for thevirtual character (S420), transmitting the thermal feedback start signal(S430), and performing a thermal feedback output operation according tothe thermal feedback start signal, which may include outputting athermal feedback corresponding to the thermal feedback type information(S440).

Hereinafter, each step of the above-described embodiment will bedescribed in more detail.

The content reproduction device 1200 may execute a game that includesthe virtual character performing emotional expression or that providesat least one text being selectable by the user (S410). The controller1260 may execute the game.

When it is detected that the emotional expression event or the textproviding event has occurred, the content reproduction device 1200 maygenerate the thermal feedback start signal including the thermalfeedback type information corresponding to the emotional expression orthe text selection (S420). The controller 1260 may detect the emotionalexpression event of the virtual character during the game progress. Forexample, the virtual character's emotional expression event may occur byperforming a specific action or making a specific facial expression. Theemotional expression event may occur according to the user's input. Theemotional expression event of the virtual character may also occur in aspecific situation during the characters' conversation. The textproviding event may correspond to a dialog action between the playablecharacter and the non-playable character or by the story progress duringthe game.

When the emotional expression event or the text providing event occurs,the controller 1260 may determine a type of the emotion expressed by thecharacter or a type of the text selected in response to the textproviding event. The controller 1260 may determine a type of the thermalfeedback according to the type of the determined emotion or the type ofthe selected text. The controller 1260 may determine the type of thethermal feedback by referring to the table for the emotion/text that isstored in the memory 1240 or stored in the game program. The table foremotion/text may define the type of thermal feedback depending on theemotion state and/or the text type.

FIG. 52 is a diagram related to the text presentation event provided inthe fourth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure.

Referring to FIG. 52, during a conversation between the player'scharacter and the non-player character, the texts are presented foruser's selection. The controller 1260 may receive a user input from theinput device via the communication module 1220 to select a particulartext, and thus may determine the type of the thermal feedback.

The content reproduction device 1200 may transmit the thermal feedbackstart signal (S430). The controller 1260 may transmit the thermalfeedback start signal including the thermal feedback type information tothe feedback device 1600 through the communication module 1220.

The feedback device 1600 may perform the thermal feedback outputoperation in accordance with the thermal feedback start signal, and mayoutput the thermal feedback corresponding to the thermal feedback typeinformation (S440).

When the thermal feedback start signal is received, the feedbackcontroller 1645 may apply a power to the thermoelectric couple array1643 to perform the thermal feedback output operation. The feedbackcontroller 1645 may generate a power supplying signal according to theinformation included in the thermal feedback start signal. For example,the controller 1260 may determine whether to perform a heat generatingoperation, a heat absorbing operation, or a thermal grill operation byreferring to the type of the thermal feedback.

According to the above-described embodiment, the user's game immersionlevel may be improved by providing a thermal feedback suitable for theemotion of the virtual character, such as outputting a hot feedback whenthe virtual character is angry and outputting a cold feedback when it isscared. Similarly, when the virtual character selects a text, a suitablethermal feedback may be provided for each text selection in the gamesuch as outputting a hot feedback for making a good decision andoutputting a cold feedback for making an evil decision, therebyimproving a user's game immersion.

4.5 Fifth Implementation

The fifth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting thermal feedbackassociated with a moving speed in a game during a game. Here, a movingspeed may mean the moving speed of the playable character in a virtualspace provided in the game.

FIG. 53 is a flowchart of the fifth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 53, the fifth implementation of the thermal feedbackproviding method may include executing a game in which a movablecharacter is implemented within a virtual space (S510), generating athermal feedback start signal including intensity informationcorresponding to a moving speed of the movable character (S520),transmitting the thermal feedback start signal (S530), and performing athermal feedback output operation in accordance with the thermalfeedback start signal, which may include outputting thermal feedbackcorresponding to thermal feedback information (S540).

The content reproduction device 1200 may execute the game thatimplements the virtual space and the movable character therein (S510).The controller 1260 may execute the game including the virtual space andthe movable character therein. The character may be a virtual charactercontrolled by the user of the game content in the virtual space of thegame. Although the virtual character is displayed (output) on the screenin the third person game, only a part of the virtual characters may ormay not be displayed on the screen in the first person game.Alternatively, in a game using a virtual reality technique, a body of auser or an input device may be regarded as the character.

The content reproduction device 1200 may generate the thermal feedbackstart signal including the intensity information of the thermal feedbackcorresponding to the moving speed of the character during the movement(S520). The controller 1260 may detect the movement of the character inthe virtual space of the game. For example, the movement of thecharacter may be made according to a user input instructing a movementof the character. The controller 1260 may obtain the user input forinstructing the character movement from the input device via thecommunication module (1220), thereby moving the character ormanipulating the movement of the character accordingly. In addition, thecharacter may move on a ride provided in the virtual space of the game.For example, in a game such as a racing game or a flight simulation, thecharacter may be represented by an airplane or an automobile that movesaccording to a user's manipulation. The controller 1260 may determinethe intensity of the thermal feedback according to the moving speed ofthe character in the virtual space. For example, the controller 1260 mayincrease the intensity of the thermal feedback as the moving speed ofthe character becomes faster in the virtual space. When the intensity isdetermined, the controller 1260 may generate the thermal feedback startsignal including the intensity information of the thermal feedback.

The content reproduction device 1200 may transmit the thermal feedbackstart signal (S530). The controller 1260 may transmit the thermalfeedback start signal including the intensity information of the thermalfeedback to the feedback device 1600 via the communication module 1220.

The feedback device 1600 may perform the thermal feedback outputoperation in accordance with the thermal feedback start signal, and mayoutput the thermal feedback corresponding to the intensity informationof the thermal feedback (S540).

When the thermal feedback start signal is received, the feedbackcontroller 1645 may apply a power to the thermoelectric couple array1643 to perform the thermal feedback output operation. The feedbackcontroller 1645 may generate a power supplying signal according to theinformation included in the thermal feedback start signal. For example,the controller 1260 may control the intensity of the thermal feedback byadjusting the voltage value of the power supply referring to theintensity information of the thermal feedback.

In this implementation, the cold feedback may be mainly used as thethermal feedback. In some embodiments, a hot feedback may be temporarilyoutput when the movement of the character is stopped. The controller1260 may determine whether the movement of the character is terminatedor not, and generate a thermal feedback start signal including a thermalfeedback information for instructing a hot feedback and transmit thethermal feedback start signal to the feedback device 1600 through thecommunication module 1220.

4.6 Sixth Implementation

The sixth implementation of a thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting thermal feedbackassociated with a health points of a character during a game.

FIG. 54 is a flowchart of the sixth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 54, the sixth implementation of the method forproviding a thermal feedback may include executing a game in which avirtual character having health points is implemented (S610), generatinga thermal feedback start signal including thermal feedback informationcorresponding to at least one of a change of the health points, aremaining physical health points, and a ratio of the remaining healthpoints to a maximum health points of the character (S620), transmittingthe thermal feedback start signal including the thermal feedbackinformation (S630), and performing a thermal feedback output operationaccording to the thermal feedback start signal, and which may includeoutputting the thermal feedback corresponding to the thermal feedbackinformation (S640).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game in which acharacter having a health points appears (S610). The controller 1260 mayexecute the game including the character operated by the user. Healthpoints may be assigned to the character. The health points may beinformation indicating a life force (vitality) of the character in thegame. For example, if character's health points are exhausted, thecharacter may die in the game. The health points may be increased ordecreased depending on events in the game. For example, the healthpoints may be decreased if the character is hit, and the health pointsmay be increased if the character uses a healing skill or a recoveryitem such as health portion.

The content reproduction device 1200 may generate the thermal feedbackstart signal including the thermal feedback information corresponding toat least one of a change of the health points, a remaining physicalhealth points, and a ratio of the remaining health points to a maximumhealth points of the character (S620).

When the health points is changed during the game progress, thecontroller 1260 may generate the thermal feedback information accordingto the change of the health points.

In one example, the controller 1260 may determine a type of the thermalfeedback depending on whether the health points is decreased orincreased. For example, the controller 1260 may determine the type ofthermal feedback as a cold feedback when the health points aredecreased. For another example, the controller 1260 may determine thetype of the thermal feedback as a hot feedback when the health points isincreased.

In another example, the controller 1260 may determine an intensity ofthe thermal feedback according to a change amount of the health points.The larger the change amount of the health points, for example, thestronger the intensity of the thermal feedback.

In another example, the controller 1260 may determine the intensity ofthe thermal feedback according to the ratio of the change amount of thehealth points to the maximum health points. The larger the ratio of thechange amount of the health points to the maximum health points, forexample, the stronger the intensity of the thermal feedback.

Further, the controller 1260 may determine the thermal feedbackinformation according to the remaining health points. The smaller theremaining health points, for example, the stronger the intensity of thethermal feedback health points. As another example, the controller 1260may determine a type of the thermal feedback based on a value ofremaining health points. For example, as the thermal feedback type, thecold feedback may be determined when the value of the remaining healthpoints is within a first range, and a hot feedback may be determinedwhen the value is within a second range.

The controller 1260 may determine the thermal feedback informationaccording to the ratio of the remaining health points to the maximumhealth points. The lower the ratio, for example, the stronger theintensity of the thermal feedback. As another example, the controller1260 may determine the type of the thermal feedback according to thehealth ratio. For example, as the thermal feedback type, the coldfeedback may be determined when the health ratio is within a firstrange, and the hot feedback may be determined when the health ratio iswithin a second range.

As described above, once the thermal feedback information is determined,the controller 1260 may generate the thermal feedback start signalincluding the determined thermal feedback information.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S630). Thecontroller 1260 may transmit the thermal feedback start signal includingthe thermal feedback information to the feedback device 1600 via thecommunication module 1220.

The feedback device 1600 may perform the thermal feedback outputoperation in accordance with the thermal feedback start signal, and mayoutput thermal feedback corresponding to the thermal feedbackinformation (S640).

When the thermal feedback start signal is received, the feedbackcontroller 1645 may apply a power to the thermoelectric couple array1643 to perform the thermal feedback output operation. The feedbackcontroller 1645 may generate a power supplying signal according to theinformation included in the thermal feedback start signal.

Accordingly, when the change in the health points occurs, the feedbackdevice 1600 may output a different thermal feedback depending on thechange amount of the health points, the ratio of the change amount tothe maximum health points, the remaining health points and/or the ratioof the remaining health points to the maximum health points.

In the above description, the thermal feedback information may bedetermined to correspond to the health points. Alternatively, thethermal feedback information may correspond to the other points used inthe game. For example, in the case of a racing game, the thermalfeedback may be associated with fuel amount points instead of the healthpoints. As another example, the thermal feedback may be associated withmana (magic) points. That is, this implementation is comprehensivelyapplicable to a variable resource points in relation to the character inthe game.

When a plurality of resources in association with the character areprovided, a different type of the thermal feedback may be assigned toeach resource. For example, the controller 1260 may execute a game foroperating the character having the resource points in the game includinga health points and a mana points. The controller 1260 may also generatea thermal feedback information corresponding to at least one of changeamount of the resource points, increase and decrease thereof, theremaining amount thereof, and the ratio of the remaining points to themaximum points. The controller 1260 may also transmit a thermal feedbackstart signal including the generated information to the feedback device1600 via the communication module 1220. The controller 1645 then mayoutput a thermal feedback operation in accordance with the thermalfeedback information. The controller 1260 may determine a type of thethermal feedback to correspond to a type of the changed resource points.For example, as the thermal feedback, a hot feedback may be determinedfor health points, and a cold feedback may be determined for manapoints.

According to the above-described implementation, the information about avital condition (health points) of the character in the game is providedto a user through the thermal feedback, thereby helping a user'sintuitive understanding for the game and improving the immersion in thegame.

4.7 Seventh Implementation

The seventh implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user based on a user input timingassociated with a timing action during a game.

FIG. 55 is a flowchart of the seventh implementation of a method forproviding a thermal feedback according to an embodiment of the presentdisclosure.

Referring to FIG. 55, the seventh implementation of the thermal feedbackproviding method may include executing a game in which a timing actionis provided (S710), generating a thermal feedback start signal includinga thermal feedback information corresponding to the user input timinginput during performing of the timing action (S720), transmitting thethermal feedback start signal including the thermal feedback information(S730), and performing a thermal feedback output operation in responseto the thermal feedback start signal, which may include outputting athermal feedback corresponding to the thermal feedback information(S740).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute the game in which thetiming action is provided, in which the result is based on the userinput timing (S710).

The controller 1260 may execute the game. The game executed in thisimplementation may include the timing action, the results of which arebased on a user input timing.

FIG. 56 is a diagram of a game providing a timing action according tothe seventh implementation of a method for providing a thermal feedbackaccording to an embodiment of the present disclosure.

FIG. 56 is a screen of a game in which an arrow can be shot in a virtualspace, based on a timing action. In the game according to FIG. 56, thedistance that the arrow flies may be increased in proportion to the timethat the user pulls back on the bow. When the user presses the button towhich the shooting action is assigned, the action of pulling the bowstarts. When the button is continuously pressed, the string of the bowis continuously and gradually pulled back. The arrow is shot from thebow at the moment of releasing the button. In FIG. 56, a gauge (whichreflects the extent to which the string of the bow is pulled back) isprovided so that the user can visually check the duration of the timingaction. Another examples of a timing action may include a ball shootingaction that continuously increases an intensity and/or accuracy of ashot during a soccer game. Another examples of a timing action mayinclude the playing of a virtual piano with a correct rhythm.

The content reproduction device 1200 may generate the thermal feedbackstart signal including the thermal feedback information corresponding tothe input timing at the time of performing the timing action (S720).

As an example, the controller 1260 may count the elapsed time from thetime point when a timing action begins during game progress, anddetermine the intensity of the thermal feedback according to the elapsedtime. The larger the elapsed time, for example, the larger the thermalfeedback intensity.

As another example, the controller 1260 may determine the thermalfeedback intensity according to a difference between a predeterminedreference value and an elapsed time. The smaller the difference, forexample, the greater the intensity of the thermal feedback.

As another example, the controller 1260 may determine the thermalfeedback intensity according to whether a length of the elapsed time isover a predetermined reference length. The larger the elapsed time, forexample, the greater the thermal feedback intensity is output until thelength is over the predetermined reference length, at which point thethermal feedback would no longer be output.

As another example, the controller 1260 may determine a type of thethermal feedback depending on whether the elapsed time has exceeded apredetermined reference value. For example, a hot feedback may be outputbefore the elapsed time exceeds the predetermined reference value, and acold feedback may be output after the elapsed time exceeds thepredetermined reference value.

After the thermal feedback information is determined, the controller1260 may generate the thermal feedback start signal including thethermal feedback information.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S730). Thecontroller 1260 may transmit the thermal feedback start signal includingthe thermal feedback information to the feedback device 1600 via thecommunication module (1220).

The feedback device 1600 may perform the thermal feedback outputoperation according to the thermal feedback start signal, and may outputthe thermal feedback corresponding to the thermal feedback information(S740). When the thermal feedback start signal is received, the feedbackcontroller 1645 may apply a power to the thermoelectric couple array1643 to perform the thermal feedback output operation. The feedbackcontroller 1645 may generate a power-supplying signal according to thethermal feedback information included in the thermal feedback startsignal.

According to the above-described implementation, the input timing may besensed by a user through the thermal feedback, thereby increasing theintuitiveness of the game.

4.8. Eighth Implementation

The eighth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackaccording to attributes of a virtual space in a game.

FIG. 57 is a flowchart of the eighth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 57, the eighth implementation of the thermal feedbackproviding method may include executing a game in which a temperatureproperty (temperature attribute) is assigned to a virtual space (S810),in response to an entrance to the virtual space, generating a thermalfeedback start signal including a thermal feedback informationcorresponding to the temperature property of the virtual space (S820),transmitting the thermal feedback start signal (S830), and performing athermal feedback output operation in accordance with the thermalfeedback start signal, which may include outputting a thermal feedbackcorresponding to the thermal feedback information (S840).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game in which atemperature property is assigned to a virtual space (S810). Thecontroller 1260 may execute the game. The virtual space in the game maybe given a temperature property. For example, an area-temperatureproperty table may be stored in the memory (1240) or a game program. Inthe area-temperature property table, a first temperature property may bematched to a first area of the virtual space and a second temperatureproperty may be matched to a second area of the virtual space.

FIG. 58 is a view showing a virtual space in a game in the eighthimplementation of the thermal feedback providing method according to anembodiment of the present disclosure.

Referring to FIG. 58, the virtual space in the game may include a flamearea, a glacier area, a plains area, a lake area, and the like as aglobal area. The flame area may be given a hot-temperature property asthe temperature property, and the glacier area may be given acold-temperature property.

The content reproduction device 1200 may generate the thermal feedbackstart signal including a thermal feedback information corresponding to atemperature property of the virtual space (S820). The thermal feedbackstart signal may be generated in response to a player character enteringto the virtual space.

The controller 1260 may determine a current area where a playercharacter is currently located by checking a location of the characterin the virtual space in virtual reality game. The controller 1260 maydetermine a current area where a user is currently located by checking alocation of the user in an augmented virtual space in an augmentedreality game. The determination may be made in real time. Alternatively,the controller 1260 may determine an entry area when a playablecharacter or a user enters a specific area in the virtual space. If thecurrent area or the entry area is determined, the controller 1260 maydetermine the thermal feedback information according to the temperatureproperty assigned to the corresponding area (the current area or theentry area) with reference to the area-temperature property table.

As an example, a type of thermal feedback may be determined according tothe temperature property. For example, the type of the thermal feedbackmay be determined as a hot feedback in the case of that a hottemperature property is assigned to the corresponding area and bedetermined as a cold feedback in the case of that a cold temperatureproperty is assigned to the corresponding area.

As another example, an intensity of the thermal feedback may bedetermined according to the temperature property. For example, thetemperature property may include a value corresponding to thetemperature value given to the corresponding area, and the controller1260 may determine the intensity of the thermal feedback according tothe value.

Once the intensity of the thermal feedback is determined, the controller1260 may generate a thermal feedback start signal including thermalfeedback intensity information.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S830), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, and may output athermal feedback corresponding to the thermal feedback information(S840).

The controller 1260 may transmit the thermal feedback start signalincluding the thermal feedback information to the feedback device 1600via the communication module 1220. When the thermal feedback startsignal is received, the feedback controller 1645 may apply a power tothe thermoelectric couple array 1643 to perform a thermal feedbackoutput operation. The feedback controller 1645 may generate a powersupplying signal according to the information included in the thermalfeedback start signal.

Furthermore, the virtual space may be classified in a hierarchicalstructure. An upper class area may include a lower class area. Forexample, the virtual space may be a global area which is the largestarea class, a local area included in the global area as a sub-area ofthe global area, a sub-local area included in the local area as asub-area of the local area, and the like. Referring to FIG. 58, a flamearea (global area) may include a volcanic sub area (local area), a lakearea (global area) may include a mountain area (local area) and a lakearea (local area), a plains area (global area) may include a lake area(local area) and a village area (local area), and a glacier area (globalarea) may include a hot spring area (local area) and a village area(local area). If the flame area, the plains area, the lake area, and theice area are defined as the local area, the above sub-areas may bedefined as the sub-local area.

Here, when the player (the character of the user) is in the glacierarea, a cold feedback may be output as the thermal feedback, but whenthe player is in the hot spring area which is the sub-area of theglacier area, a hot thermal feedback may be output as the thermalfeedback. That is, the temperature property assigned to the sub area(lower class area) may be prioritized to the temperature propertyassigned to the upper class area. If the player exits from the hotspring area, a cold feedback may be output as the thermal feedback. Incase of that the same temperature property is assigned to the upperclass area and the lower class area (for example, the flame area and thevolcanic sub area), a weaker feedback may be output to the user for theupper class area and a stronger feedback may be output for the lowerclass area. For example, a weak hot feedback may be assigned to theflame area, and a strong hot feedback may be assigned to the volcanicsub area.

According to the above-described implementation, when a playablecharacter enters the virtual space in the virtual reality or when theuser enters the virtual space in the virtual reality or the augmentedreality, a thermal feedback suitable for a virtual environment of thearea may be output to provide a realistic thermal feedback to the user.4.9. Ninth Implementation

The ninth implementation of a thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting thermal feedbackaccording to a shooting event in the game.

FIG. 59 is a flowchart of the ninth implementation of a thermal feedbackproviding method according to an embodiment of the present disclosure.

Referring to FIG. 59, the ninth implementation of the thermal feedbackproviding method may include executing a game using a physics enginesupporting a collision event and a trigger event for processing ashooting event (S910), generating a thermal feedback start signalincluding thermal feedback information determined based on whether theprocessing is the collision event or the trigger event when a hit eventoccurs to a playable character (or a user) (S920), transmitting thethermal feedback start signal (S930), and performing a thermal feedbackoutput operation in accordance with the thermal feedback start signal,which may include outputting a thermal feedback corresponding to thethermal feedback information (S940).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game using a physicsengine supporting a collision event and a trigger event for processing ahit event by a virtual object (S910).

The controller 1260 may execute the game. The game executed in thepresent implementation may be a game for processing a hit event using aphysics engine.

The physics engine is software used to simulate physical phenomena inthe fields of computer graphics, video games, movies, and the like. Thephysics engine is mainly used in the above-mentioned fields forrealizing image processing more realistically. The physics engine mayalso be provided as a middleware for computing physical phenomena inreal time on the game field. Representative functions of the physicsengine may include the processing of collisions between virtual objects.

A hit event occurs when a playable character operated by a user or auser in a virtual reality and/or augmented reality gets hit by a virtualobject. In this implementation, the game may process the hit event as acollision event or a trigger event using the physics engine.

Specifically, the physics engine may assign the collision determinationattribute to the virtual object. The collision determination attributemay include a collision attribute and a trigger attribute. In a virtualspace implemented by a physics engine, a virtual object assigned by thecollision attribute is processed so as to be able to collide withanother virtual object. In contrast, the virtual object assigned by thetrigger attribute is processed so as not to collide with another virtualobject.

Thus, when two virtual objects are located at the same coordinates inthe virtual space, if the attributes of both virtual objects are thecollision attributes, the physics engine may treat them as a collisionevent. Alternatively, if at least one of the two virtual objects has thetrigger attribute, the physics engine may treat it as a trigger event.Here, in case of the collision event, the bodies of two virtual objectsare not overlapped with each other and react as if they collide witheach other as in the real world. In contrast, according to the triggerevent, the spaces occupied by the two virtual objects overlap eachother, so that one object can pass through another object.

A playable character or the user of the virtual reality typically hasthe collision attribute. If a virtual object having the collisionattribute hits the player's character, the physics engine may generatethe collision event. On the other hand, the physics engine may generatea trigger event if the object hitting the playable character has thetrigger attribute.

For example, when a playable character falls onto a ground from a highplace in the virtual reality and/or augmented reality, the collisionevent may occur because the ground is given the collision attribute.Similarly, since other characters or virtual vehicles in the game alsohave the collision attribute, they can cause collision events when theyhit the user's playable character. In another example, in a shootinggame, a bullet is typically given the trigger attribute, so a triggerevent occurs when the character or other virtual object is hit by thebullet.

The content reproduction device 1200 may generate a thermal feedbackstart signal including a thermal feedback information determineddepending on whether the processing is the collision event or thetrigger event when a hit event occurs to a playable character (or auser) by a virtual object in a virtual a virtual reality game (and/or aaugmented reality) (S920).

When a hit event occurs in the game, the controller 1260 may determinewhether the hit event is a collision event or a trigger event. This canbe determined according to the collision determination attribute of thehit object. The controller 1260 may determine an intensity of thethermal feedback in a different manner for the collision event and thetrigger event.

In one example, the controller 1260 may determine the intensity of thethermal feedback based on an amount of impulse calculated in the physicsengine if the hit event is determined to be a collision event. Forexample, the larger the impulse amount, the stronger the intensity ofthe thermal feedback. The amount of the impulse in the physics enginecan be calculated based on the relative speed between the virtualobjects and the mass value given to the virtual objects. That is, as themass of the impulse object is large and the relative speed is high, theintensity of the thermal feedback may be determined to be strong.

As another example, when the hit event is determined as the triggerevent, the controller 1260 may determine the intensity of the thermalfeedback based on a type of a virtual object or a projectile, or thespeed of a virtual object, since the physics engine does not calculatethe amount of impulse at the time of the trigger event. For example, ifhit by a bullet having a trigger attribute, the controller 1260 maydetermine that the higher the absolute velocity of the bullet, thestronger the intensity of the thermal feedback. For another example, ifhit by a bullet with the trigger attribute, the controller 1260 maydetermine the intensity of the thermal feedback according to a type ofbullet. For another example, if hit by a bullet with a triggerattribute, the controller 1260 can determine the intensity of thethermal feedback according to a type of weapon that fired the bullet.The controller 1260 may refer to a table in which the thermal feedbackintensity is set for each type of the weapon. For example, the strongerthe weapon is, the stronger the intensity of thermal feedback is. To setthe intensity of the thermal feedback depending on the type of theprojectile or the weapon, the controller 1260 can refer to the table inwhich the thermal feedback intensity is set for each type of projectileor weapon.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S930), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, which may beoutputting a thermal feedback corresponding to the thermal feedbackinformation (S940).

The controller 1260 may transmit the thermal feedback start signalincluding the thermal feedback information to the feedback device 1600via the communication module 1220. When the thermal feedback startsignal is received, the feedback controller 1645 may apply a power tothe thermoelectric couple array 1643 to perform the thermal feedbackoutput operation. The feedback controller 1645 may generate the powersupplying signal according to the information included in the thermalfeedback start signal.

According to the above-described implementation, a realistic thermalfeedback may be output based on a hit event occurring in a virtualspace. In the case of being hit by the virtual object having a collisionattribute, the intensity of the thermal feedback could be adjusted usingthe amount of impulse calculated by the physics engine, therebyproviding a thermal feedback corresponding to the actual reality. Inaddition, in cases where the amount of impulse is difficult tocalculate, other appropriate parameters may be used instead of theamount of impulse to realistically provide the thermal feedbackcorresponding to the hit event.

4.10. Tenth Implementation

The tenth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackassociated with a heat transfer attribute of a virtual object in a game.

FIG. 60 is a flowchart of the tenth implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 60, the tenth implementation of a thermal feedbackproviding method may include executing a game including a virtual objectto which a heat transfer attribute are assigned (S1010), generating athermal feedback start signal including a thermal feedback informationconsidering the heat transfer attribute (S1020), transmitting thethermal feedback start signal (S1030), and performing a thermal feedbackoutput operation in accordance with the thermal feedback start signal,which may include outputting a thermal feedback corresponding to thethermal feedback information (S1040).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game including avirtual object to which heat transfer attributes are assigned (S1010).

The controller 1260 may execute the game. In this implementation, thegame includes the virtual object, and the virtual object may be given aheat transfer attribute. The heat transfer attribute may indicate howthe virtual object delivers heat to a player (character) and/or a user.

The virtual object with the heat transfer attribute may be treated as aheat source in the game. The heat source may include a conduction heatsource and a radiant heat source. The conductive heat source cantransfer a heat only when substantially contacted with the user (orplayable character) in the game. The radiant heat source can transfer aheat even when away from the user (or player's character) in the game.

The content reproduction device (1200) may generate a thermal feedbackstart signal including a thermal feedback information in considerationof the heat transfer attribute of the virtual object (S1020).

The controller 1260 may generate the thermal feedback start signalincluding the thermal feedback information in consideration of thethermal transfer attribute of the virtual object.

For example, if the heat transfer attribute of the virtual object is theconductive heat source, the controller 1260 may determine whether a user(or a playable character) and a virtual object are substantially incontact with each other. The controller 1260 may generate the thermalfeedback start signal when the conductive heat source contacts with theuser (or playable character). The controller 1260 may determine a typeand an intensity of the thermal feedback based on the temperatureattribute assigned to the conductive heat source. Conversely, thecontroller 1260 may not generate a thermal feedback start signal if theuser (or player's character) is apart from the conductive heat source.

In another example, if the heat transfer attribute of the virtual objectis a radiant heat source, the controller 1260 may generate a thermalfeedback start signal even when the user (or the playable character) isspaced apart from the virtual object. The controller 1260 may determinea type of the thermal feedback based on the temperature attributeassigned to the radiant heat source (the virtual object). The controller1260 may also determine an intensity of the thermal feedback accordingto the temperature attribute assigned to the radiant heat source (thevirtual object).

Alternatively, when determining the thermal feedback information for theradiant heat source, the controller 1260 may further consider a type ofradiant heat source and a distance between the user (or player'scharacter) and the radiant heat source.

A radiant heat sources may be divided into several types depending on arelationship between a distance (between the radiant heat source and theplayable character (a user) and an amount of the heat transferredtherewith. For example, the radiant heat source may include an emissiveheat source, a directional heat source, and an areal heat source. Theemissive heat source is a heat source that the transmitted heat amountbecomes smaller as the distance from the heat source increases. Thedirectional heat source is a heat source that transfers a constant heatregardless of the distance from the heat source. The areal heat sourceis a heat source that transfers heat only to a region within apredetermined distance from a heat source and does not transfer heat toa region beyond the predetermined distance. Also, the areal heat sourcemay be set to transfer a constant heat, or to transfer a smaller heat asthe distance increases from the center of the area.

Accordingly, in the case where the radiant heat source is the emissiveheat source, the controller 1260 may set the thermal feedback intensityto be stronger as the distance to the user (or the player's character)becomes closer. When the radiant heat source is the directional heatsource, the controller 1260 may set the thermal feedback intensity onlyconsidering the temperature of the heat source regardless of thedistance from the user (or the player's character). When the radiantheat source is an areal heat source, the controller 1260 may determinethe thermal feedback intensity according to the temperature of the heatsource when the user (or the player character) is within a predeterminedregion from the heat source. The thermal feedback intensity may also bedetermined in consideration of the distance of the heat source to theuser (or the player's character) and the temperature of the heat sourceas described above. Moreover, the thermal feedback start signal may benot generated when the user (or the player's character) is outside thepredetermined region.

To calculate the heat transferred by the heat source in the game, thegame of this implementation may use the physics engine described above.

When the thermal feedback information is determined, the controller 1260may generate the thermal feedback start signal including the thermalfeedback information.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S1030), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, and may output athermal feedback corresponding to the thermal feedback information(S1040).

The controller 1260 may transmit the thermal feedback start signalincluding the thermal feedback information to the feedback device (1600)via the communication module 1220. When the thermal feedback startsignal is received, the feedback controller 1645 may apply a power tothe thermoelectric couple array 1643 to perform a thermal feedbackoutput operation. The feedback controller 1645 may generate a powersupplying signal according to the information included in the thermalfeedback start signal.

According to the above-described implementation, the heat transferred tothe user or the playable character may be calculated according to thedistance and other information related to the heat source on the virtualspace, and the corresponding thermal feedback may be output to give arealistic thermal experience to the user.

4.11. Eleventh Implementation

The eleventh implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by associating and outputtingthermal feedback associated with a thermal conductivity of a virtualobject in a game.

FIG. 61 is a flowchart of the eleventh implementation of the thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 61, the eleventh implementation of a thermal feedbackproviding method may include executing a game including a virtual objectto which a surface attribute (surface property) related to thermalconduction is assigned (S1110), generating a thermal feedback startsignal including a thermal feedback information determined based on thesurface attribute when a touch event or a grip event occurs (e.g., whenthe user or playable character touches or grabs the object) (S1120),transmitting the thermal feedback start signal (S1130), and performing athermal feedback output operation in accordance with the thermalfeedback start signal, which may include outputting a thermal feedbackcorresponding to the thermal feedback information (S1140).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may play a game including a virtualobject to which a surface attribute related to thermal conduction isassigned, in the eleventh implementation of the thermal feedbackproviding method (S1110).

The content reproduction device 1200 may present a game including thevirtual object to which the surface attribute related to thermalconduction is assigned, in the eleventh implementation of the thermalfeedback providing method (S1110).

The content reproduction device 1200 may generate a thermal feedbackstart signal including a thermal feedback information. The thermalfeedback information is determined in association with the surfaceattribute of the virtual object upon a touch event or a grip eventinvolving the virtual object (S1120).

The controller 1260 may generate the thermal feedback start signal whenthe touch event or the grip event occurs. The controller 1260 maydetermine the thermal feedback information to be included in the thermalfeedback start signal.

As an example, a type of thermal feedback may be determined based on thetemperature (temperature attribute) assigned to the virtual object. Forexample, the type of the thermal feedback may be selected among a hotfeedback, a cold feedback and a thermal grill feedback based on thetemperature assigned to the virtual object. The controller 1260 may alsodetermine an intensity of the thermal feedback based on the temperatureof the heat source (virtual object).

When the thermal feedback information is determined, the controller 1260may generate the thermal feedback start signal including the thermalfeedback information.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S1030), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, and output thethermal feedback corresponding to the thermal feedback information(S1040).

The controller 1260 may transmit the thermal feedback start signalincluding the thermal feedback information to the feedback device 1600via the communication module 1220. When the thermal feedback startsignal is received, the feedback controller 1645 may apply a power tothe thermoelectric couple array 1643 to perform the thermal feedbackoutput operation. The feedback controller 1645 may generate a powersupplying signal according to the information included in the thermalfeedback start signal.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including a thermal feedback information (S1130), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, and output thethermal feedback corresponding to the thermal feedback information(S1140).

The controller 1260 may transmit the thermal feedback start signalincluding the thermal feedback information to the feedback device 1600via the communication module 1220. When the thermal feedback startsignal is received, the feedback controller 1645 may apply a power tothe thermoelectric couple array 1643 to perform the thermal feedbackoutput operation. The feedback controller 1645 may generate a powersupplying signal according to the information included in the thermalfeedback start signal.

4.12. Twelfth Implementation

The twelfth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackbased on the priority of a thermal event that occurs in a game.

In a multimedia environment including games, the thermal events (forexample, the hit event, the touch event, the grip event or entry into aspecific area) may occur at the same time.

The thermal event means an event that induces an output of thermalfeedback. Examples of the thermal event may include a change in thehealth points (sixth embodiment), entering into a thermal attributeassigned area (eighth embodiment), and getting hit or shot (ninthembodiment), or the like

For example, a hit event (a shooting event) inducing a hot feedback mayoccur in a state in which a playable character (or a user) enters aspecific area inducing a cold feedback.

The content reproduction device 1200 may determine the priority betweenthe thermal events, and output a thermal feedback for one thermal eventwhich has a higher priority than the other thermal event.

For example, when a bullet inducing a hot feedback is hit on thecharacter or a user in a lake area inducing a cold feedback, thepriority on the thermal events may be determined. In the case where apriority for the hit event by the bullet is determined as being higherthan a priority for the lake-entering event, the hot feedback includedby the hit event will be output first.

Specifically, when the hit event inducing the hot feedback occurs in thestate where a character or a user enters a specific region for inducingthe cold feedback, the controller 1260 may generate a thermal feedbackstart signal including a thermal feedback information, wherein thethermal feedback information may be configured to output the thermalfeedback for the concurrently-occurred thermal events with prioritygiven to outputting the hot feedback due to the hit event rather thanthe cold feedback due to entry into the specific region, until theeffect of the hit event is terminated. Then, the controller 1260 maytransmit the thermal feedback start signal to the feedback device 1600through the communication module 1220 so that the feedback controller1645 performs a feedback output operation according to the thermalfeedback information. In addition, in the above case, after the effectfor the thermal event having a higher priority (for example, the hitevent) is terminated, the other thermal feedback induced by the otherthermal event (for example, the entry into the specific region) may beprovided or resumed. For example, when thermal feedback based on an areaevent is output, and an object event occurs in this state, thermalfeedback based on the object event is output. When the object eventcomes to an end, the thermal event based on the area event may beresumed.

4.13. Thirteenth Implementation

The twelfth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure described aboverelates to which thermal event should be prioritized when thermal eventsoccur at the same time.

However, the thermal feedback associated with two concurrent thermalevents may be combined.

For example, if a hot feedback and a cold feedback are outputsimultaneously, a thermal grill feedback can be output. If the intensityof the hot feedback is stronger than the intensity determined based onthe neutral ratio, the content reproduction device 1200 may output a hotgrill feedback, and in the opposite case, the content reproductiondevice 1200 may output a cold grill feedback.

That is, the controller 1260 may generate a thermal feedback startsignal such that a hot feedback induced by one thermal event and a coldfeedback induced by the other thermal event are simultaneously providedto the user, thereby outputting the thermal grill feedback (cold thermalgrill feedback or hot thermal grill feedback). Further, the controller1260 may determine a type of the thermal grill feedback by comparing theintensity of the hot feedback induced by the one thermal event with theintensity of the cold feedback induced by the other thermal event. Forexample, when the intensity of the hot feedback is stronger, the type ofthermal grill feedback may be determined as a hot thermal grillfeedback. When the intensity of the cold feedback is stronger, the typeof the thermal feedback may be determined as a cold thermal grillfeedback.

The controller 1260 then may transmit the thermal feedback start signalgenerated through the communication module 1220 to the feedback device1600. The feedback device 1600 may refer to the thermal feedbackinformation included in the thermal feedback start signal, and performan output operation.

4.14. Fourteenth Implementation

The fourteenth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackassociated with consideration of a thermal resistance of a characterwhen a thermal event occurs in a game.

FIG. 62 is a flow chart of the fourteenth implementation of a thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 62, the fourteenth implementation of a thermalfeedback providing method may include executing a game in which a playercharacter or a player in a virtual space has a thermal resistance(S1410), generating a thermal feedback start signal including a thermalfeedback intensity information determined considering the thermalresistance (S1420), transmitting a thermal feedback start signal(S1430), and performing a thermal feedback output operation inaccordance with the thermal feedback start signal, which may includeoutputting a thermal feedback corresponding to the thermal feedbackinformation (S1440).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game in which aplayer character or a player in the virtual space has a thermalresistance (S1410).

The controller 1260 may execute the game. Here, in the game executed inthis implementation, the thermal resistance may be given to the playercharacter or the user avatar in the virtual reality. The thermalresistance may be assigned to the character or avatar in the form of acapacity or ability. In addition, if the character or avatar is equippedwith equipment such as weapon or armor, a total thermal resistance ofthe character may be determined by adding the capacity of the thermalresistance of the equipment to the thermal resistance of the characteror avatar.

The content reproduction device 1200 may generate a thermal feedbackstart signal including thermal feedback intensity information determinedin consideration of the thermal resistance (total thermal resistance) atthe time of occurrence of a thermal event (S1420).

The controller 1260 may process the thermal feedback informationaccording to the thermal event when the thermal event occurs. If thecharacter is given the thermal resistance, the controller 1260 mayadjust the intensity of the thermal feedback considering the thermalresistance of the character. For example, the higher the thermalresistance, the lower the intensity of the thermal feedback.

Alternatively, the thermal resistance may be classified into a hotresistance/a cold resistance/a pain resistance. In this case, thecontroller 1260 may determine a type of the thermal feedback induced bythe thermal event occurrence, select a type of thermal resistance to beconsidered according to the type of thermal feedback, and adjust theintensity of the thermal feedback according to the selected thermalresistance.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including thermal feedback information (S1430), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal and output a thermalfeedback corresponding to the thermal feedback information (S1440).

The controller 1260 may transmit the thermal feedback start signalincluding thermal feedback information to the feedback device 1600 viathe communication module 1220. When the thermal feedback start signal isreceived, the feedback controller 1645 may apply a power to thethermocouple array 1643 to perform the thermal feedback outputoperation. The feedback controller 1645 may generate the power supplyingsignal according to the information included in the thermal feedbackstart signal.

According to the above-described implementation, since the intensity ofthe thermal feedback is adjusted according to the thermal resistance inthe virtual space, a thermal feedback may be provided that reflects thedegree of equipment and growth of characters in a role playing game(RPG).

4.15. Fifteenth Implementation

The fifteenth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackassociated with consideration of the size of an image related to athermal event displayed in a field of view (FOV) when the thermal eventoccurs in a game.

FIG. 63 is a flowchart of the fifteenth implementation of a method forproviding a thermal feedback according to an embodiment of the presentdisclosure.

Referring to FIG. 63, the fifteenth implementation of a method forproviding a thermal feedback may include executing a game including athermal event (S1510), generating a thermal feedback start signalincluding an intensity information which is determined according to thearea occupied by the image related to the thermal event occurred in theFOV (S1520), transmitting the thermal feedback start signal (S1530), andperforming a thermal feedback output operation in accordance with thethermal feedback start signal, which may include outputting a thermalfeedback corresponding to the thermal feedback information (S1540).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may execute a game including athermal event (S1510), which may be performed by the controller 1260.Also, the controller 1260 may output a video signal during the game,which may be a video signal according to the FOV of the virtual camera1480 in the virtual space.

The content reproduction device 1200 may generate a thermal feedbackstart signal including thermal feedback intensity information determinedaccording to an area occupied by an image related to a thermal eventwhen the thermal event occurs in the FOV (S1520).

The controller 1260 may calculate the area occupied by the image relatedto the thermal event occurs in the FOV. The controller 1260 may thendetermine the thermal feedback intensity information according to theoccupied area and include it to generate a thermal feedback startsignal. For example, as the number of pixels occupied by an imageassociated with the thermal event increases, the intensity of thethermal feedback may be increased.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S1530), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal and may output athermal feedback corresponding to the thermal feedback information(S1540).

The controller 1260 may transmit a thermal feedback start signalincluding thermal feedback information to the feedback device 1600 viathe communication module 1220. When the thermal feedback start signal isreceived, the feedback controller 1645 may apply a power to thethermocouple array 1643 to perform a thermal feedback output operation.The feedback controller 1645 may generate the power supplying-signalaccording to the information included in the thermal feedback startsignal.

According to the above-described embodiment, even though the samethermal event is generated, a strong thermal feedback may be output whenthe user zooms-in on the visual field in the thermal event direction,and weak thermal feedback can be output when the user zooms-out.Similarly, the thermal feedback may be weakened by turning the FOV suchthat the user moves the thermal event out of the FOV and may be enhancedby turning the FOV such that the thermal event enters into the FOV.Accordingly, the user's thermal feedback can be further improved. 4.16.Sixteenth Implementation

The sixteenth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method ofproviding a thermal feedback to a user by outputting a thermal feedbackassociated with the augmented reality in a game.

FIG. 64 is a flow chart of the sixteenth implementation of a thermalfeedback providing method according to an embodiment of the presentdisclosure.

Referring to FIG. 64, the sixteenth implementation of a thermal feedbackproviding method may include capturing an image for a real world(S1610), detecting a thermal object in a user's FOV based on thecaptured image (S1620), generating a thermal feedback informationaccording to the detected thermal object (S1630), transmitting a thermalfeedback start signal including the thermal feedback information(S1650), and performing a thermal feedback output operation inaccordance with the thermal feedback start signal, which may beoutputting a thermal feedback corresponding to the thermal feedbackinformation (S1650).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may capture an image of the realworld through a camera 1480 (S1610). The content reproduction device1200 may be provided in the form of an HMD including a camera 1480.Alternatively, the content reproduction device 1200 may be provided as aconsole device being associated with an HMD. In this case, contentreproduction device may obtain (receive) the image (video) of the realworld obtained by a camera of the HMD through a communication module1220.

The content reproduction device 1200 may detect a thermal object in theFOV of the user based on the image (S1620). The controller 1260 mayidentify the thermal object included in the image through an imagerecognition algorithm. The thermal object may be a real object or avirtual object inserted into the real world by the augmented realityapplication.

The content reproduction device 1200 may generate a thermal feedbackinformation according to the detected thermal object (S1630). Thecontroller 1260 may obtain a temperature attribute (temperatureproperty) assigned to the thermal object using the identification resultof the thermal object. Further, the controller 1260 may determine a typeof thermal feedback according to the temperature attribute.

The content reproduction device 1200 may transmit the thermal feedbackstart signal including the thermal feedback information (S1640), and thefeedback device 1600 may perform a thermal feedback output operation inaccordance with the thermal feedback start signal, which may beoutputting a thermal feedback corresponding to the thermal feedbackinformation (S1650).

The controller 1260 may transmit the thermal feedback start signalincluding thermal feedback information to the feedback device 1600 viathe communication module 1220. When the thermal feedback start signal isreceived, the feedback controller 1645 may apply a power to thethermocouple array 1643 to perform a thermal feedback output operation.The feedback controller 1645 may generate a power-supplying signalaccording to the information included in the thermal feedback startsignal. 4.17. Seventeenth Implementation

The seventeenth implementation of the thermal feedback providing methodaccording to an embodiment of the present disclosure is a method forproviding a thermal feedback to a user by outputting a thermal feedbackassociated with considering a field of view (FOV) or a characterposition where a thermal event occurs, and a position of the thermalevent in the screen.

The present implementation will be described based on the case where auser uses a feedback devices 1600 in both hands.

FIG. 65 is a flowchart of the seventeenth implementation of a method forproviding thermal feedback according to an embodiment of the presentdisclosure.

Referring to FIG. 65, the seventeenth implementation of the thermalfeedback providing method may include reproducing a multimedia content(S1710), generating a thermal event during the reproduction of themultimedia content (S1720), determining a thermal feedback target inaccordance with a direction of occurrence of the thermal event (S1730),transmitting a thermal feedback start signal in accordance with thedetermination of the thermal feedback target (S1740), and performing athermal feedback output operation in response to the reception of thethermal feedback start signal, which may be outputting a thermalfeedback corresponding to the thermal feedback information (51750).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The content reproduction device 1200 may reproduce the multimediacontent (S1710). The multimedia content may include at least one thermalevent. The content reproduction device 1200 may also communicate with afirst feedback device 1600 which is gripped by a right hand of a userand a second feedback device 1600 which is gripped by a left hand of theuser.

The content reproduction device 1200 may generate a thermal eventaccording to the multimedia content reproduction (S1720). The thermalevent may include any of the thermal events described in the abovethermal feedback providing methods.

When a thermal event occurs during the reproduction of the multimediacontent, the content reproduction device 1200 may determine a thermalfeedback target according to a direction of the thermal event (S1730).

The controller 1260 may determine the direction of occurrence of athermal event. Specifically, in the case of the first-person viewmultimedia content including the virtual reality application or theaugmented reality application, the direction of the occurrence of thethermal event may be determined based on a relationship between aposition of the thermal event and the FOV. In the case of third-personview content, the direction of the thermal event may be determined basedon the position of the player character in the game.

When the direction of the occurrence of the thermal event is determined,the controller 1260 may determine the thermal feedback target accordingto the generation direction.

FIG. 66 is a view showing a thermal feedback target object according toan occurrence point of a thermal event in a first-person game accordingto the seventeenth implementation of the thermal feedback providingmethod.

For example, referring to FIG. 66, when a thermal event occurs in thecentral part of the FOV in the first-person-view application, thecontroller 1260 may determine a thermal feedback target to be both thefirst feedback device 1600 and the second feedback device 1600. Also,when a thermal event occurs on the right side based on the FOV, thecontroller 1260 may determine the thermal feedback target to be thefirst feedback device 1600. Alternatively, when a thermal event occurson the left side based on the FOV, the controller 1260 may determine thethermal feedback target to be the second feedback device 1600.

The content reproduction device 1200 may transmit a thermal feedbackstart signal in accordance with the determination of the thermalfeedback target (S1740).

Once the thermal feedback target is determined, the controller 1260 maytransmit a thermal feedback start signal via the communication module1220 to the feedback device 1600 which is the determined thermalfeedback target.

When the thermal feedback start signal is received, the feedback device1600 may perform a thermal feedback output operation in response to thereception of the thermal feedback start signal. The thermal feedbackoutput may be output based on the thermal feedback information (51750).

Alternatively, in this implementation, as the point of occurrence of thethermal event moves, the feedback device 1600 outputting the thermalfeedback may be changed. For example, if a user turns the FOV while athermal event occurs on the right side of the FOV, the point where thethermal event occurs can move through the center to the left side. Thethermal feedback target move from the right feedback device 1600, toboth feedback devices 1600, to the left feedback device 1600.

Although the present embodiment has been described above as beingperformed in an environment using a plurality of feedback devices 1600,it is not necessarily so. For example, this implementation may beapplied when a single feedback device 1600 is used. For example, ifthere is a plurality of thermal output modules 1640 arranged in onegaming controller gripped by both hands, the feedback controller 1645 ofthe feedback device 1600 may determine which of the first thermal outputmodule 1640 and the second thermal output module 1640 included in thegaming controller may be selected as the target, and the thermal outputmodule 1640 may output the thermal feedback accordingly. As anotherexample, the present implementation may be applied even in the case ofthe feedback device 1600 having only one thermal output module 1640. Forexample, the only one module may include a thermocouple array 1643constituted by a plurality of thermocouple groups 1644 (that is, the onethermal module may be configured to be controllable by region), thefeedback controller 1645 of the feedback device 1600 may determine whichregion (the center/right/left/whole) of the module will be designated asthe target of the thermal feedback and the thermal output module 1640may output the thermal feedback accordingly.

The methods of providing thermal feedback according to embodiments ofthe present disclosure described above can be used alone or incombination with each other. In addition, since each of the stepsdescribed in each thermal feedback providing method is not essential,the method of providing thermal feedback can be performed by includingall or part of the steps. Also, since the order in which the steps aredescribed is merely for convenience of explanation, the steps in themethod of providing thermal feedback are not necessarily performed inthe order described.

Also, in the method of providing thermal feedback according to anembodiment of the present disclosure described above, any steps notdescribed as being executed by a specific controller may be performed byone or both of the application controller and the feedback controller1645 of the feedback device 1600. In addition, in the above description,the steps described as being performed by the application controller maybe performed by the feedback controller 1645, and vice versa. Inaddition, steps described above as being performed by one of theapplication controller or the feedback controller 1645 maybe performedby the collaborative operation of both controllers. As alreadymentioned, the application controller and feedback controller 1645 maybe implemented as a single controller 1260.

5. A Method for Generating Multimedia Content

Hereinafter, a method of generating multimedia content used in thethermal feedback providing method will be described. The method ofgenerating multimedia content according to an embodiment of the presentdisclosure may include a method of generating video content and a methodof generating a game or a sensory application.

5.1. An Electronic Device

The method for generating multimedia content according to an embodimentof the present disclosure may be performed by the electronic device2000. For example, the electronic device 2000 may include a PC, aworkstation, a laptop, notebook, tablet PC, smart phone, and the like.

FIG. 67 is a block diagram of an electronic device 2000 according to anembodiment of the present disclosure.

Referring to FIG. 67, the electronic device 2000 may include an inputmodule 2200, an output module 2400, a memory 2600, and a controller2800.

The input module 2200 may receive user input from a user. The user inputmay be in various forms including keyboard input, touch input, andphonetic input, and the like. Examples of the input module 2200 mayinclude a conventional type of a keypad, a keyboard, and a mouse, atouch sensor for sensing a user's touch, and other various known typesof input devices. Also, the input module 2200 may be implemented in theform of an input interface (USB port, PS/2 port, etc.) that connects anexternal input device receiving the user input with the electronicdevice, instead of a device sensing the user input in itself.

The output module 2400 may output various information and provide it toa user. The output module 2400 may include, e.g., a display foroutputting video, a speaker for outputting sounds, a haptic device forgenerating vibration, and other various known types of output devices.In addition, the output module 2400 may be implemented in the form of aport-type output interface that connects the above-described individualoutput devices to the electronic device 2000.

The memory 2600 may store various kinds of information. The memory maystore data temporarily or semi-permanently. Examples of the memoryinclude a HDD, a SSD, a flash memory, a ROM, and a RAM. The memory 2600may be provided in a form embedded in the electronic device 2000 or in adetachable form. The memory (2600) may store various data needed or usedfor driving the electronic device 2000, including the OS data fordriving the electronic device 2000.

The controller 2800 controls an overall operation of the electronicdevice (2000). That is, the controller 2800 may perform calculations andprocesses of various information and control the operation of othercomponents of the electronic device. The controller may be implementedin a computer or similar device depending on the hardware, software or acombination thereof. In hardware, a controller may be provided in theform of an electronic circuit, such as a microprocessor, that performs acontrol function with an electrical signal processing. In software, thecontroller may be provided in a form of a program that drives a hardwarecontroller. The operation of the electronic device 2000 may beinterpreted as being performed by the controller 2800, unless otherwisenoted in implementations of the method of generating multimedia contentproviding the thermal feedback described below.

5.2. First Implementation

Hereinafter, the first implementation of a method for generatingmultimedia content according to an embodiment of the present disclosurewill be described. This embodiment relates to a method of generatingvideo content among multimedia content.

In the first implementation of the thermal feedback providing methoddescribed above, in the case of reproducing the video content set sothat the thermal event scene reproduction time point coincides with oneof the thermal feedback reproduction, the thermal feedback starting timepoint in the content reproduction device being earlier than the thermalfeedback reproduction time point so that the user can feel the thermalfeedback at the time point of reproducing the video corresponding to thethermal event.

Alternatively, the reproduction time point of the thermal feedback maybe set in advance considering the time difference between the powerapplying-time point and the user feeling the thermal feedback. Themethod for generating multimedia content providing a thermal feedbackaccording to an embodiment of the present disclosure relates to a methodfor generating thermal feedback data of video content to be linked tovideo or audio, even if the thermal feedback providing system 1000performs the thermal feedback output operation at the reproduction timeof the thermal feedback according to the thermal feedback data of thevideo content.

FIG. 68 is a flowchart of the first implementation of the method forgenerating multimedia content according to an embodiment of the presentdisclosure.

In FIG. 68, the first implementation of the method for generatingmultimedia content may include obtaining a reproduction time point of athermal event scene (S1810), calculating a thermal feedback reproductiontime point based on a reproduction time point and a correction time ofthe thermal event scene (S1820) and generating thermal feedback databased on the thermal feedback regeneration point (S1830).

Hereinafter, each step of the above-described implementation will bedescribed in more detail.

The electronic device 2000 may obtain the output time point of thethermal event scene (S310). The controller 2800 may receive a user inputrelating to a reproduction timing of a thermal event scene to be linkedto thermal feedback through the input module 2200.

The electronic device 2000 may provide a UI screen that allows a user tobe able to easily input a reproduction time point of a thermal eventscene through an output module 2400 such as a display. The UI screen maybe provided somewhat similar to the screen provided by the conventionalimage editing program.

The UI screen may include a progressive bar reflecting the reproductiontime point of the video content, an indicator disposed on theprogressive bar, and a video window for outputting a scene correspondingto the time point indicated by the indicator. The UI screen may alsoinclude a control panel for controlling a reproduction of the videocontent. The user can move the indicator on the progressive bar orreproduce the video content using the control panel, and in doing so,the user can see the scene output through the video window during thisprocess. When a scene desired to be a thermal event scene is displayedat the video window, the user can select a time point as thereproduction time point of the thermal event scene, at which theindicator on the progressive bar is located corresponding to the scene.The UI screen may be provided along with a menu for further selectingthe object, type, strength and duration of the thermal feedback.

Via the UI screen, the user can search a scene to be linked with thermalfeedback through the video window and select a time point of outputtinga desired scene in the video window. The controller 2800 may obtain theoutput time point of the specific scene by receiving the user inputthrough the input module 2200.

The electronic device 2000 may calculate the power applying-time pointfor the thermal feedback based on the output time point and thecorrection time of the thermal event scene (S1820).

The controller 2800 may determine the correction time as describedabove. The controller 2800 may further calculate a time point to applythe power by subtracting the correction time from the obtained outputtime point.

When the power applying-time point is calculated, the electronic device2000 may generate thermal feedback data based on this time point(S1830). The thermal feedback data may include information on the typeand intensity of thermal feedback, and information on the powerapplying-time point for starting the output of the thermal feedback. Thethermal feedback data may further include information about a targetfeedback device to which the thermal feedback is to be output asdescribed above.

The thermal feedback data generated in this way can be used to outputthermal feedback linked to the video output, which is referred to whenreproducing the video content. For this, the thermal feedback data maybe provided as a separate file from a video file, such as a subtitlefile providing subtitles during the video output. Alternatively, it maybe provided as a single file in which thermal feedback data and videodata are integrated.

As in this embodiment, when the thermal feedback data is generated inconsideration of the delay time, the content reproduction device 1200does not need to correct the output time point of the thermal feedbackto account for the delay time, which advantageously reduces thecomputations needed to reproduce the multimedia content.

Alternatively, since the delay time may be slightly different dependingon the manufacturer of the feedback device 1600, the correction time canbe adaptively processed for each situation in consideration of the typeof the feedback device 1600.

In the above description of the implementation, the video content amongthe multimedia content is explained, but the embodiment can be appliedto other types of multimedia content such as a game or a sensoryapplication. Typically, the embodiment may be used for a scene in whicha video cut scene appears in the course of a game, and may be the sameas the first implementation of the thermal feedback providing method.

In this embodiment and the first embodiment of the thermal feedbackproviding method, the effect described above may be somewhat small whenthe response time of the thermoelectric element is fast. However, incase of treating the contact surface 1641 with a material such as rubberto improve the grip feeling of the user in the gaming controller, thetime required for heat transfer from the thermoelectric element to thecontact surface 1641 may become longer, and in such a case, theadvantage is expected to be further exerted.

5.3. Second Implementation

Hereinafter, the second implementation of a method for generatingmultimedia content according to an embodiment of the present disclosurewill be described. This embodiment relates to a method of generatingmultimedia content provided in the form of a game or a sensoryapplication of multimedia content.

The second implementation of the thermal feedback providing methoddescribed above may include generating a virtual object and assigningthermal characteristics to the generated virtual object.

The virtual object refers to an object capable of interacting with aplayer inside a game, a sensory game application, or the like.

When a virtual object is generated, the generated virtual object can begiven a thermal characteristic. Thermal characteristics (attributes,properties) are directly or indirectly related to the type and intensityof thermal feedback.

In addition, various thermal properties (attributes) may be defined.Some examples are temperature values, material values, resistancevalues, and the like. In concrete examples, all the information used inthe determination of the intensity/type/target device of the thermalfeedback may be the thermal characteristics of the object in the thermalfeedback providing method.

In an example, the virtual object is given an element property so thatthe type of thermal feedback can be determined accordingly. This isexplained in detail in the third implementation of the thermal feedbackproviding method.

In another example, a heat transfer method may be determinedcorresponding to the thermal property (attribute) assigned to theobject, which is illustrated in the tenth embodiment of a thermalfeedback providing method.

Also, this implementation may be utilized on a game production engine.For example, an existing game production engine may include a physicsengine that processes object collision and light paths, and a thermalprocessing functionality may be added to such physics engine.

FIG. 69 is a flowchart of the thermal feedback providing methodaccording to an embodiment of the present disclosure.

In step S1910 the reproduction content device 1200 may execute a game.For example, the controller 1260 may execute the game. The game mayinclude virtual areas and virtual objects.

In step 1920, controller 1260 may detect an area event. An area eventmay reflect that a player character, which is controlled by a user,enters an area. In some embodiments, the is a local area. In otherembodiments, the area is a global area.

In step 1930, controller 1260 may send a signal to feedback device 1600to output a thermal feedback corresponding the area in which the playercharacter enters.

In step 1935, controller 1260 may determine whether the player characterleft the area. If the player character leaves the area (step S1935:Yes), controller 1260 may finalize the process. However, if the playercharacter does not leave the area (step S1935: No), controller 1260 maycontinue to step 1940.

In step 1940, controller 1260 may detect an object event. In someembodiments, an object event may represent that a virtual object isinfluencing the player's character. In other embodiments, the objectevent may represent a collision between player's character and a virtualobject.

In step 1950, controller 1260 may send a signal to feedback device 1600to output a thermal feedback corresponding the object event. In someembodiments, the thermal feedback corresponding the object event may bebased on the area event. For example, if player character is in a hotarea, such as the flame area, and the object is a fire bolt, the thermalfeedback corresponding the object event may have a low intensity.However, if the player character is in a hot area and the object is anice bolt, the thermal feedback corresponding the object event may have ahigh intensity.

In step 1955, controller 1260 may determine whether the virtual objectstops influencing the player's character. If the virtual object stopsinfluencing player's character, controller 1260 may return to step 1930.However, if the virtual object continues influencing player's character,controller 1260 may continue to step 1960.

In step 1960, controller 1260 may detect a new area event. The new areaevent may reflect that the player's character enters a new area. In someembodiments, the new area may be different from the area of step 1920.In other embodiments, the new area may be within the area of step 1920.

In step 1970, controller 1260 may send a signal to feedback device 1600to output a thermal feedback corresponding the new area in which theplayer character enters. The thermal feedback corresponding the new areamay be determined based on the object event. For example, if the newarea is a lake and the object event is a fire bolt, the thermal feedbackcorresponding the new area may be low intensity cold. However, if thenew area is a volcano and the object is a fire bolt, the thermalfeedback corresponding the new area may be high intensity hot.

In step 1975, controller 1260 may determine whether the player leavesthe new area. If the player leaves the new area controller 1260 mayreturn to step 1950. However, if the player does not leave the new area,controller 1260 may return to step 1970.

The multimedia content generation methods providing the thermal feedbackaccording to an embodiments of the present disclosure described abovecan be used alone or in combination with each other. In addition, sinceeach of the steps described in each multimedia content generation methodis not essential, the multimedia content generation method can beperformed by including all or part of the steps. Also, since the orderin which the steps are described is merely to facilitate explanations,the steps in the method of providing thermal feedback are notnecessarily performed in the order described. Non-dependent steps may beperformed in any order or in parallel.

The foregoing description is merely illustrative of the technical ideaof the present disclosure and various changes and modifications may bemade without departing from the essential characteristics of the presentdisclosure by those skilled in the art. Therefore, the embodiments ofthe present disclosure described above may be implemented separately orin combination.

Therefore, the embodiments disclosed in the present disclosure areintended to illustrate rather than limit the scope of the presentdisclosure, and the scope of the technical idea of the presentdisclosure is not limited by these embodiments. The scope of protectionof the present disclosure should be construed according to the followingclaims, and all technical ideas within the scope of equivalents thereofshould be construed as being included in the scope of the presentdisclosure.

1. A content reproduction device, wherein the content reproductiondevice reproduces a multimedia content and cooperates with at least onefeedback device outputting a thermal feedback using a thermoelectricelement, including: a memory storing data; a communication modulecommunicating with the feedback device; and a controller configured to:execute a virtual reality application providing a virtual space, whereinthe virtual space includes a virtual area to which a temperatureattribute is assigned, the virtual area having a global area and a localarea included in the global area, when a player character enters theglobal area, control, via the communication module, a feedback device tooutput the thermal feedback corresponding to the temperature attributeof the global area, wherein the feedback device outputs the thermalfeedback using a thermoelectric element performing a thermoelectricoperation, and when the player character enters the local area, control,via the communication module, the feedback device to override thethermal feedback corresponding to the temperature attribute of theglobal area with the thermal feedback corresponding to the temperatureattribute of the local area.